Performing operations without requiring split mirrors in a multi-class file system

ABSTRACT

System and method for performing operations in a multi-class file system without requiring split mirrors. For one or more storage classes in the multi-class file system, operations that require stable copies of the storage classes may be performed without using split mirrors. In one embodiment, read-only data may be assigned and/or migrated to lower storage classes, and operations may be performed on the read-only storage classes without using split mirrors. In one embodiment, to perform an operation without using a split mirror, a write lock of a storage class may be examined to determine if the write-locked storage class has been written to during the operation on the storage class and, if so, the operation may be retried for the storage class. In one embodiment, the file system software may be blocked from enabling a storage class for writing for the duration of the operation on the storage class.

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention is related to the field of computer systems and, moreparticularly, to data storage systems.

2. Description of the Related Art

In data storage environments such as corporate LANs, total storage needsare increasing and storage costs are an increasing part of the ITbudget. More and/or higher capacity storage devices may be added, butthis solution is expensive and difficult to manage, and does not addressthe root of the problem. There is a limit to storage capacity no matterhow much storage capacity is added. This solution tends to provide aconstant cost per byte for storage, as it tends not to take advantage oflower cost-per-byte storage devices. A high percentage of data stored ina storage environment may be infrequently accessed, or never accessed atall after a certain time. Lower cost-per-byte for storage may berealized using methods that move at least some of this infrequentlyaccessed data off more expensive storage devices and on to the lessexpensive storage devices.

Hierarchical Storage Management (HSM) is a data storage solution thatprovides access to vast amounts of storage space while reducing theadministrative and storage costs associated with data storage. HSMsystems may move files along a hierarchy of storage devices that may beranked in terms of cost per megabyte of storage, speed of storage andretrieval, and overall capacity limits. Files are migrated along thehierarchy to less expensive forms of storage based on rules tied to thefrequency of data access.

In HSM systems, data access response time and storage costs typicallydetermine the appropriate combination of storage devices used. A typicalthree tier HSM architecture may include hard drives as primary storage,rewritable optical as secondary storage, and tape as tertiary storage.Alternatively, hard drives may be used for secondary storage, and WORM(Write Once, Read Many) optical may be used as tertiary storage.

Rather than making copies of files as in a backup system, HSM migratesfiles to other forms of storage, freeing hard disk space. Events such ascrossing a storage threshold and/or reaching a certain file “age” mayactivate the migration process. As files are migrated off primarystorage, HSM leaves stubs to the files on the hard drive(s). These stubspoint to the location of the file on the alternative storage, and areused in automatic file retrieval and user access. The stub remainswithin the file system of the primary storage, but the file itself ismigrated “offline” out of the file system onto the alternative storage(e.g. tape).

In HSM, when a file that has been migrated to a lower rank of storage,such as tape, is accessed by an application, the stub may be used toretrieve and restore the file from the lower rank of storage. The fileappears to be accessed by the application from its initial storagelocation, and demigration of the file back into the file system isperformed automatically by the HSM system using the stub. While on thesurface this demigration may appear transparent to the user, in practicethe process of accessing and restoring the file from offline storage(e.g. tape) may introduce a noticeable time delay (seconds, minutes, oreven hours) to the user when compared to accessing files stored onprimary storage. Thus, accessing offloaded data in an HSM system istypically non-transparent to the application or user because of thedifference in access time. In addition, since HSM introduces asubstantial time lag to access offloaded data, HSM systems typicallyonly offload low access (essentially, no access) data.

A file system may be defined as a collection of files and file systemmetadata (e.g., directories and inodes) that, when set into a logicalhierarchy, make up an organized, structured set of information. Filesystems may be mounted from a local system or remote system. File systemsoftware may include the system or application-level software that maybe used to create, manage, and access file systems.

File system metadata may be defined as information file system softwaremaintains on files stored in the file system. File system metadata mayinclude definitions and descriptions of the data it references.Generally, file system metadata for a file includes path information forthe file as seen from the application side and corresponding file systemlocation information (e.g. device:block number(s)). File system metadatamay itself be stored on a logical or physical device within a filesystem.

File systems may use data structures such as inodes to store file systemmetadata. An inode may be defined as a data structure holdinginformation about files in a file system (e.g. a Unix file system).There is an inode for each file, and a file is uniquely identified bythe file system on which it resides and its inode number on that system.An inode may include at least some of, but is not limited to, thefollowing information: the device where the inode resides, lockinginformation, mode and type of file, the number of links to the file, theowner's user and group IDs, the number of bytes in the file, access andmodification times, the time the inode itself was last modified and theaddresses of the file's blocks on disk (and/or pointers to indirectblocks that reference the file blocks).

When performing operations such as backup, periodic replication orcertain batch jobs file system cloning or device level split mirrors aretypically used to achieve a stable copy of the storage. This isprimarily used with hardware arrays. Many software split mirrormechanisms support space saving implementations that do not haveoverhead for read-only data. However, hardware split mirrors typicallydo not support space saving implementations, or may have to do complexconfiguration steps even for space saving split mirrors.

SUMMARY

Embodiments of a system and method for performing operations in amulti-class file system without requiring split mirrors are described. Asplit mirror is a stable copy broken off from a mirror image. Whenperforming some operations such as backup, periodic replication orcertain batch jobs file system cloning or device level split mirrors aretypically used to achieve a stable copy of the storage. Using amulti-class file system implemented by a multi-class storage mechanismto segregate data being written from data being read, such operationsthat normally use split mirrors may be performed without having togenerate split mirrors for all the data.

A multi-class file system including a hierarchy of storage classes maybe implemented on one or more storage devices. In one embodiment, thestorage classes may be ordered in the hierarchy of storage classesaccording to performance characteristics from a highest storage classincluding one or more high-performance storage devices to a loweststorage class comprising one or more low-performance storage devices. Inone embodiment, one or more of the storage classes may be write-lockedstorage classes for read-only data.

Actively written to files may be assigned and/or migrated to higherstorage classes and read-only files may be assigned and/or migrated towrite-locked lower storage classes in the hierarchy. In one embodiment,the file system may migrate portions of a file to different storageclasses. For example, actively written records of a database file may bestored in a higher storage class of the multi-class file system, withinactive database records migrated to a write-locked lower storageclass.

An operation that requires a stable copy of the storage classes may needto be performed, e.g. by the file system software or by an applicationusing the multi-class file system. The operation may, for example, be animage backup, creating a second copy for doing testing or batchprocessing on a database, or any other operation that requires a stablecopy. To perform the operation on a read-only storage class, a writelock of the storage class may be examined to determine if thewrite-locked storage class has been written to. If the write-lockedstorage class has not been written to, the operation may be performed onthe write-locked storage class without generating a split mirror of thewrite-locked storage class. In one embodiment, the entity performing theoperation may block the file system software from enabling thewrite-locked storage class for writing for the duration of theoperation.

BRIEF DESCRIPTION OF THE DRAWINGS

The following detailed description makes reference to the accompanyingdrawings, which are now briefly described.

FIG. 1 illustrates file system software including a multi-class storagemechanism implementing a multi-class file system according to oneembodiment.

FIG. 2 illustrates file system software including a multi-class storagemechanism implementing a multi-class file system and assigning andmigrating data within the file system according to one embodiment.

FIG. 3 illustrates a multi-class file system based on a multi-devicefeature of File System software according to one embodiment.

FIG. 4 illustrates a multi-class file system based on multipleindependent file systems under a virtualization layer according to oneembodiment.

FIG. 5 illustrates a multi-class file system in a network storageenvironment according to one embodiment.

FIG. 6 is a flowchart of a method for assigning and migrating files orportions of files in a multi-class file system according to oneembodiment.

FIGS. 7A through 7E illustrate the migration process in an exemplarymulti-class file system with three storage classes according to oneembodiment.

FIGS. 8A through 8D illustrate an exemplary metadata structure for datathat may be modified during assignment and migration of data in amulti-class file system according to one embodiment.

FIG. 9 illustrates a backup mechanism for an exemplary multi-class filesystem on a host system in a network storage environment according toone embodiment.

FIG. 10 illustrates a backup mechanism for an exemplary multi-class filesystem with three storage classes according to one embodiment.

FIGS. 11A-11D illustrate an exemplary migration and backup schedule foran exemplary multi-class file system according to one embodiment.

FIG. 12 is a flowchart of a method for performing backups of amulti-class file system according to one embodiment.

FIG. 13 illustrates performing an operation on an exemplary multi-classfile system that requires stable images of the storage classes withoutusing split mirrors for one or more storage classes according to oneembodiment.

FIG. 14 is a flowchart of a method for performing operations in amulti-class file system without requiring split mirrors according to oneembodiment.

While the invention is described herein by way of example for severalembodiments and illustrative drawings, those skilled in the art willrecognize that the invention is not limited to the embodiments ordrawings described. It should be understood, that the drawings anddetailed description thereto are not intended to limit the invention tothe particular form disclosed, but on the contrary, the intention is tocover all modifications, equivalents and alternatives falling within thespirit and scope of the present invention as defined by the appendedclaims. The headings used herein are for organizational purposes onlyand are not meant to be used to limit the scope of the description orthe claims. As used throughout this application, the word “may” is usedin a permissive sense (i.e., meaning having the potential to), ratherthan the mandatory sense (i.e., meaning must). Similarly, the words“include”, “including”, and “includes” mean including, but not limitedto.

DETAILED DESCRIPTION OF EMBODIMENTS

Embodiments of a system and method for implementing a file system acrossdifferent classes of storage in a storage system are described.Embodiments may define multiple classes of storage and automatically andtransparently migrate data between the storage classes within the samefile system to meet the usage needs of the business. This data may bemigrated as files or portions of files, and may include, but is notlimited to, application data and/or file system metadata. Embodimentsmay be used to transparently adapt the cost of storage to the usagepatterns of data on the storage, thereby reducing the amount thatcustomers need to spend for their storage. In embodiments, storagedevices may be classified into two or more different classes of storageto implement a multi-class file system. Embodiments may provide amulti-class storage mechanism for managing and implementing sets ofuser-defined policies for assigning and migrating data within themulti-class file system.

FIG. 1 illustrates file system software including a multi-class storagemechanism implementing a multi-class file system according to oneembodiment. File system software 102 may include, but is not limited to,an embodiment of the multi-class storage mechanism 104 and software thatprovides “traditional” File System functionality 106. File Systemfunctionality 106 may be provided by any of various File Systemimplementations including, but not limited to, disk-based file systemssuch as Veritas' VxFS and UNIX File System (UFS), and networked filesystem such as Network File System (NFS) and AT&T's Remote File Share(RFS).

File system software 102 may implement, on one or more storage devices,a multi-class file system 120. In one embodiment, the software thatprovides the File System functionality 106 may be used to implement themulti-class file system 120, while the multi-class storage mechanism 104manages the policies and performs assignment and migration of datawithin the multi-class file system 120 according to the policies. Themulti-class file system 120 may include one or more physical and/orlogical storage devices organized in a hierarchy of two or more storageclasses 122 according to one or more characteristics of the storagedevices and/or data requirements of the user(s) of the file system. Themulti-class file system 120 is made available to the application(s) 100as one file system. The storage classes 122 of the multi-class filesystem 120 are transparent to the application(s) 100.

In embodiments, the multi-class storage mechanism 104 may make anautomatic and transparent determination on where to initially place afile (i.e., in which storage class 122) according to the policies. Themulti-class storage mechanism 104 may later relocate or migrate the file(or a portion of the file) to another storage class 122 according to thepolicies. A migrated file remains online within the file system, even ifthe file is moved to the lowest storage class 122; migration does notmove files to offline storage.

In a multi-class file system 120, the same file system has multiplestorage classes 122. Each storage class 122 is part of a single overallfile system. As far as the user or application is concerned, the filesare stored in the same logical storage location within the file system(e.g. a directory or folder). It is transparent from the applicationand/or user's perspective whether a file is on one storage class 122 oranother, or if parts of a file are on different storage classes 122.From the application and/or user's perspective, the file appears to bein the same location in the file system where the user put it. Migrateddata remains in the file system no matter which storage class it ismigrated to. Migrated data remains active or online in the file system,and does not have to be restored from offline storage media in order tobe accessed.

In one embodiment, the number of storage classes 122 in a multi-classfile system 120 is configurable—there may be 2 to N user-defined storageclasses 122 in a multi-class file system 120. As an example, athree-class file system may be implemented, and a first portion of thedata (e.g. 5%) may be classified as most active data and may be placedin a highest storage class, a second portion of the data (e.g. 15%) maybe classified as less active data and may be placed in a middle storageclass, and a third portion of the data (e.g. 80%) may be classified asleast active (or inactive) data and may be placed in a lowest storageclass (without offloading the data to tape or other media). Thus,embodiments may preferably reduce cost of overall storage withoutsignificantly impacting performance, robustness, etc.

In embodiments, there may be multiple devices in the same storage class122. A storage class 122 may be based, for example, on cost/performancecharacteristics of the storage devices. A storage class 122 may, forexample, include five different devices, which may or may not be thesame make and model, but all the devices are considered in the samestorage class 122. If a file is put on a storage class 122, it may bestored to any of the devices in the storage class 122 (or alternativelyacross two or more devices in the storage class 122). In one embodiment,portions of a file may be stored or migrated to devices in two or morestorage classes 122. For example, one or more active records of adatabase may be stored in a higher storage class 122 includinghigh-performance storage devices, while less or inactive records of thedatabase may be stored in or migrated to lower storage classes 122including lower-performance storage devices.

In embodiments, storage classes 122 may be defined (e.g. by a user oradministrator) based on one or more characteristics including, but notlimited to, performance and cost of the storage. Storage devices do nothave to be exactly similar in characteristics to be placed in the samestorage class 122, but it may be preferable to place storage devicesthat are similar in one or more primary characteristics together in thesame storage class 122. For example, storage devices may be grouped intostorage classes 122 according to performance characteristics, with thefastest storage devices placed in a first (highest) storage class 122,and slower storage devices placed in lower storage classes 122. Forexample, two or more fast mirrored storage devices (which may bedifferent makes or models) may be grouped in a first storage class, twoor more RAID-5 devices may (which may be different makes or models) begrouped in a second storage class, even slower storage devices (whichmay be different makes or models) may be placed in a third storageclass, and so on.

Embodiments may allow a user to define as many storage classes 122 asdesired or required and to separate the storage classes 122 according toany characteristics the user desires or requires. In one embodiment, auser may further segment a storage class 122 into two or more subclassesof storage as desired or required. As examples, a storage class 122 maybe segmented into subclasses for two or more departments of anenterprise to separate the departments' data onto different storagedevices within the storage class, or into subclasses containingfrequently-read data and less-frequently-read data, or into subclassescontaining frequently-written data and frequently-read data, or intosubclasses containing frequently-written data andless-frequently-written data. In this embodiment, the policiesmaintained and implemented by the multi-class storage mechanism 104 mayspecify, at the storage class and/or subclass level, storage classesand/or subclasses of a storage class to which certain files or portionsof files are to be assigned and/or migrated.

Embodiments may provide business-oriented hierarchical management ofonline storage. Rather than simply migrating data to lower classes ofstorage based on how recently or frequently the data are accessed,embodiments may make migration decisions within the multi-class filesystem 120 based on one or more of other business-oriented factors.Embodiments may organize and migrate data in a multi-class file system120 using policies oriented towards the business value of the data. Thebusiness value may be based, for example, on the user, application,department, etc that originated, owns, or most frequently accesses thedata, time criticality of the data, and/or time of the data in abusiness or application cycle. Embodiments may implement classes ofstorage and place data in the storage classes 122 according to thebusiness-oriented policies. In embodiments, files or portions of filesmay be placed in a storage class 122 and/or migrated to differentstorage classes 122 according to usage parameters and/orbusiness-oriented parameters such as how valuable and/or important thedata is, as determined by the predefined policies for evaluating theparameters. Embodiments may apply policies about the value, importance,and usage of the data, or even other factors such as ownership of thedata (e.g. what department or user owns and/or accesses the data),within the multi-class file system 120 to preferably provide a lowerunderlying hardware cost with essentially the same responsiveness.

Policies used by the multi-class storage mechanism 104 may be based onfile type, users, location, department, application, importance, value,or other factors in addition to file size and usage. The factors thatpolicies may evaluate may be factors that are important to a businesssuch as how valuable, critical, or important the data is, and howcritical the response time is. For example, the policies may specifythat a set of business-critical data that may be infrequently accessedshould remain in a highest storage class, and/or that another set ofdata that is not as critical may be stored or migrated to a lowerstorage class regardless of the data's frequency of access. The data inthe lower storage classes or even the lowest storage class, still withinthe same file system, may be accessed without having to be restored tothe file system from offline storage, and may be migrated to higherstorage classes transparently to the application and/or user if thepolicies, applied by the multi-class storage mechanism 104, indicatethat migration of the files is necessary or desired.

Implementation of the policies may be performed at allocation timeand/or during use. The policies may be implemented within the FileSystem software 102, eliminating a level of interruption.

Embodiments of the multi-class storage mechanism 104 may migrate datawithin the multi-class file system 120, no matter how busy the data isor whether the data is being used for read or write, if the policiesindicate that the data is to be migrated. The multi-class storagemechanism 104 makes policy-based decisions about the importance, value,and/or usage of data and migrates the data to a storage class 122 withappropriate characteristics. The data remains online as part of the samefile system, and the migration of the data is transparent to the user.Data, including application data and file system metadata, may be movedup or down in the multi-class file system 120 transparently to the user.

In one embodiment, the multi-class storage mechanism 104 may place orrelocate parts of a file in different storage classes 122 according tothe policies. For example, one part of a file that is frequentlyaccessed may be placed in a higher storage class composed of fasterstorage devices, and another part of the file that is less frequentlyaccessed may be migrated to a lower storage class composed of slower,less expensive storage devices. Data, including application data andfile system metadata, remains active in the same file system regardlessof which storage class 122 the files or parts of files containing thedata are stored in or migrated to.

In one embodiment, file system metadata may be used to track the storageand migration of data in the multi-class file system 120. When a file orportion(s) of a file is stored in or migrated to a storage class, thefile system metadata may be modified to indicate the location of thefile or portion(s) of the file in the multi-class file system 120. Fromthe application or user's perspective, the path to the file does notchange when the metadata is modified to reflect migration of a file orportion of a file. The file system metadata may be stored and handledseparately from other data or alternatively may be handled together withother data in a storage class 122 (typically, but not necessarily, atthe highest storage class).

In one embodiment, to migrate a file or portion of a file, a range ofblocks of the file may be locked while the blocks are being migrated. Inone embodiment, the File System software 102 obtains a lock on the rangeof blocks, reads the data into system memory, writes the data out to thenew storage class location, changes the mapping in the metadata, andunlocks the range of blocks. In other embodiments, ranges of blocks maybe copied from one storage class location to another without being readinto and written out of system memory. The range of blocks may be lockedonly for the time the data is being migrated. The rest of the blocks inthe file remain unlocked during the migration of the range of blocks. Inone embodiment, the number of blocks in a range that are locked andmigrated at one time may be tuned for performance. The effects of themigration are transparent to any user of the data.

In one embodiment, to migrate a file or portion of a file, the file orportion of a file being migrated may not be locked during the transferof the data. Instead, the data may be migrated to its new location, andthe file or portion of a file may be locked only while updating themetadata. In one embodiment, an intelligent controller, array, or othermechanism may be used to migrate the data outside the host without goingthrough system memory, without locking the file or portion of a filewhile the data is being transferred, and locking the file or portion ofa file only while updating the metadata. In these embodiments, the filesystem software may need to check when it locks the file or portion of afile for the metadata update that no writes happened while the blockswere being copied. These embodiments lock the file or portion of a filefor a shorter time than locking during the migration, so are lessdisruptive to file accesses. In one embodiment, if the file or portionof a file is being written, then a migration up for the written blocksmay be performed by allocating a new set of blocks for the data beingwritten and then directly writing them to the new disk location.

In one embodiment, a storage class 122 may be added to a multi-classfile system 120 and data in files or portions of files, includingapplication data and file system metadata, may be automatically andtransparently moved to the new storage class. For example, if new,faster storage devices are available, one or more of the storage devicesmay be added to the multi-class file system 120 as a new, highest classof storage, and files or portions of files may be automatically migratedto the new, highest storage according to one or more policies. The samemay be performed for intermediate or lowest classes of storage.

The capacity of storage classes may be increased or decreased ifnecessary or desired. In one embodiment, a storage class may bemonitored for free space and automatically grown if necessary. In oneembodiment, a storage device within a storage class may be grown toincrease the capacity of the storage class. For example, a storagedevice with a total of 4 gigabytes of disk space may have 2 gigabytes ofthe disk space allocated to a storage class in a multi-class filesystem. If necessary or desired, free disk space on the storage devicemay be added to the storage class to grow the storage class. Data may beautomatically and transparently moved to the newly allocated disk spaceon the storage device according to the policies. In one embodiment, oneor more storage devices may be added to a storage class and data may beautomatically and transparently moved to the new storage deviceaccording to the policies.

In one embodiment, the migration of data in a multi-class file systemmay be controlled by the type(s) of application data and/or file systemmetadata in the multi-class file system. In other words, data may besegregated in the hierarchy of storage classes according to its type. Inone embodiment, file system metadata may be segregated into differentstorage classes based on the type of metadata and migrated betweenstorage classes as can application data.

FIGS. 2 through 8D illustrate means for implementing a multi-class filesystem comprising a hierarchy of storage classes on a plurality ofstorage devices and means for assigning and migrating data to differentstorage classes in the hierarchy of storage classes according to a setof policies for the multi-class file system.

FIG. 2 illustrates file system software including a multi-class storagemechanism implementing a multi-class file system and assigning andmigrating data within the file system according to one embodiment. Filesystem software 102 may include, but is not limited to, an embodiment ofthe multi-class storage mechanism 104 and software that provides“traditional” File System functionality 106. File System functionality106 may be provided by any of various File System implementationsincluding, but not limited to, disk-based file systems such as Veritas'VxFS and UNIX File System (UFS), and networked file system such asNetwork File System (NFS) and AT&T's Remote File Share (RFS).

File system software 102 may implement, on one or more storage devices,a multi-class file system 120. In one embodiment, the software thatprovides the File System functionality 106 may be used to implement themulti-class file system 120, while the multi-class storage mechanism 104manages the policies and performs assignment and migration of datawithin the multi-class file system 120 according to the policies. Themulti-class file system 120 may include one or more physical and/orlogical storage devices organized in a hierarchy of two or more storageclasses according to one or more characteristics of the storage devicesand/or data requirements of the user(s) of the file system. In thisexample, the multi-class file system 120 has N classes—a first storageclass 110, a second storage class 112, and so on to an Nth storage class114. The multi-class file system 120 is made available to theapplication(s) 100 as one file system. The storage class structure ofthe multi-class file system 120 is transparent to the application(s)100.

In one embodiment, each storage class of the multi-class file system 120may have different characteristics such as cost and/or geometry. In oneembodiment, the storage classes may form a hierarchy, with the mostexpensive and typically fastest storage devices at the top, and theleast expensive (and typically slower) storage devices at the bottom.The different storage classes may include heterogeneous storage devicesto help achieve cost differences. For example, first storage class 110may include one or more fast mirrored storage devices, second storageclass 112 may include one or more RAID-5 storage devices, and a loweststorage class may include one or more read/write optical storagedevices. In one embodiment, the amount of total storage at each storageclass in the hierarchy may increase at each lower storage class of thehierarchy, though this is not required.

Note that, in some implementations, other criteria than cost andperformance may be used to organize the storage devices into a hierarchyof storage classes. For example, storage capacity may be used as acriterion to select storage devices for a lower storage class. Asanother example, ownership or usage of the storage devices may be usedas a criterion to place storage devices in storage classes. In thisexample, a storage device may be placed in a storage class based on thedepartment or application that uses the storage device rather than oncost or performance characteristics of the storage device, and thepolicies may be implemented such that data belonging to that departmentor application is assigned to that storage device.

Multi-class storage mechanism 104 may store files, or portions of files,in storage classes of the multi-class file system 120 according to oneor more user-defined policies geared to the business needs of theenterprise. Multi-class storage mechanism 104 may also migrate filesand/or portions of files between storage classes in the multi-class filesystem 120 according to one or more user-defined policies. For example,as indicated at 130, files and/or portions of files may be migrated fromfirst storage class 110 to second storage class 112, or from secondstorage class 112 to first storage class 110, according to the policies.As indicated at 132, files and/or portions of file may be migrated fromsecond storage class 112 to lower storage classes in the multi-classfile system 120 (e.g. the Nth storage class 114). The policies may beconfigured to move data up or down in the hierarchy of storage classes.In one embodiment, when data is reclassified to a lower or loweststorage class, the data may be compressed to further lower the cost ofstorage. Note that migrated files and/or portions of files remain in thesame file system and are not migrated offline when migrated to any ofthe storage classes, even the lowest storage class. Also note thatmigration is performed transparently to the application(s) 100.

While FIG. 2 shows data migrating down from a higher storage class downto a next lower storage class, and data migrating up from a lowerstorage class to a next higher storage class, note that data may bemigrated from any storage class to any other storage class in thehierarchy of storage classes according to the policies. For example,when migrating data down in the hierarchy of storage classes, some datamay be migrated from the highest storage class to the next lower storageclass, and other data may be migrated directly to the lowest storageclass according to the policies. Similarly, when migrating data up inthe hierarchy of storage classes, some data may be migrated from thelowest storage class to the next higher storage class, and other datamay be migrated directly to the highest storage class according to thepolicies.

In one embodiment, the multi-class storage mechanism 104 may store newfiles in the multi-class file system at the highest or higher storageclasses of the hierarchy. In one embodiment, as data ages (dataincluding files and portions of files, e.g. a set of records in adatabase), the data may be moved down the hierarchy to lower-coststorage on lower storage classes to reduce the cost of storing the data.In one embodiment, data may be written into fast storage (e.g. fastmirrored storage) and then batched to slower storage such as RAID-5storage, for example on a nightly schedule. Note that business-relatedattributes of the data such as importance, value, or ownership may beconsidered in decisions to assign and/or migrate the data. For example,a user may define the policies so that a file or portion of a fileconsidered critical to business operations, and thus requiring fastestpossible access times, may be kept in a highest storage class althoughaccess information may indicate that the file or portion of the file bemigrated to a lower storage class. As another example, a user may definethe policies so that a file or files, or portions of files, belonging toa certain group within the business may be kept in a higher storageclass although access information may indicate that the file(s) orportions of file(s) be migrated to a lower storage class.

In one embodiment, when multi-class storage mechanism 104 migrates afile or portion of a file down to a lower storage class or up to ahigher storage class in the multi-class file system, file systemmetadata 108 (e.g., inode information) is modified to reflect the movebecause the file or portion of a file is being moved to a differentstorage device. Note that file path information exposed to theapplication(s) 100 is not modified; the move is transparent to theapplication(s) 100. Also note that the moved data remains “on-line” inthe multi-class file system 120, even when moved to the lowest storageclass. The data is not moved “offline” and replaced by a stub (e.g. asin HSMs). Thus, the metadata (e.g., inode of the file) is modified toindicate the new location (e.g. device:block number(s) ordevice:extent(s)) of the data. If an application 100 requests the data,the data is still on-line in the file system, so it can be accesseddirectly without having to restore the data from offline storage. Thus,the structure of the multi-class file system 120 is transparent toapplication(s) 100 accessing the data stored in the multi-class filesystem 120. Regardless of where a file, block, or extent is stored inthe multi-class file system 120, the file appears the same to theapplication(s) 100.

In one embodiment, the policies may be implemented to consider businesscriteria (e.g. value) and/or usage patterns of the data in makingassignment and migration decisions. In one embodiment, the policies forclassifying and migrating data in the hierarchy may be tailored tobusiness needs to lower the total cost of the storage farm withoutcompromising on performance or availability. There are many possiblecombinations of data usage and business criteria that may be implementedin the policies for use by the multi-class storage mechanism 104 inclassifying data as to its initial storage class in multi-class filesystem 120 and in reclassifying and migrating data down to a lowerstorage class or up to a higher storage class in multi-class file system120. The policies may make decisions according to the application, user,and/or business segment (e.g., department, division, etc) accessing orassociated with the data, access patterns for the data, a specifiedpolicy (e.g. files in a particular directory or of a particular businessgroup or employee could be assigned to high, medium, or low accessclasses), and/or any other desired parameter or parameters.

In one embodiment, multi-class storage mechanism 104 may track access ofdata, including application data and file system metadata, in files orportions of files, automatically downgrade at least a portion of thedata on a storage class (e.g. at a time interval specified by a user oradministrator) that has not been accessed for a given time interval, andmigrate the downgraded data, in files or portions of files, to a lowerstorage class. In one embodiment, multi-class storage mechanism 104 mayalso be configured to migrate data on a storage class that has beenaccessed to a higher storage class. In one embodiment, a policy orpolicies may be operationally driven (e.g. to migrate data up to ahigher class to do end-of-period processing, then back down when theprocessing is done).

One embodiment may break files down into smaller ranges of blocks orextents and keep statistics on the ranges (access times, usagecharacteristics, etc) to help make policy-based automatic decisions. Theranges of the files may then be separated on different classes accordingto the statistics as determined by the policies. For example, databasesmay have large files with high-priority and low-priority records mixedin. The files may be broken down into ranges that include high-priorityor low-priority records, and statistics may be kept on the individualranges. As another example, DB2 databases may have partitioned databased on certain indexes or fields in the database. Combinations ofpartitions may be created, and the DB2 database may be partitioned amongclasses of storage. This may be used, for example, in scaling clusteringsolutions.

In one embodiment, the partitioning technique used to implement amulti-class file system may be a multi-device feature of a File System(represented as File System functionality 106 in FIG. 2). FIG. 3illustrates a multi-class file system based on a multi-device feature ofFile System software according to one embodiment. File system software142 may include, but is not limited to, an embodiment of the multi-classstorage mechanism 144 and software that provides “traditional” FileSystem functionality 146. File System functionality 146 may be providedby any of various File System implementations including, but not limitedto, disk-based file systems such as Veritas' VxFS and UNIX File System(UFS), and networked file system such as Network File System (NFS) andAT&T's Remote File Share (RFS).

Each storage device in the multi-class file system 160 may be a logicalor physical storage device. Typically, a physical device will beassigned to a particular class, but it is possible that two or morelogical devices on one physical device may be allocated to differentstorage classes. The storage devices may be segmented into N storageclasses in the multi-class file system 160 (in this example, threestorage classes) by File System functionality 146. In this example, thefirst storage class includes storage device 150, the second storageclass includes storage devices 152A and 152B, and the third storageclass includes storage devices 154A, 154B, and 154C. Note that a storageclass may include any number of individual storage devices.

The storage devices may be divided into storage classes by File Systemfunctionality 146 according to characteristics of the storage devices.In one embodiment, the storage classes may be arranged from a firststorage class of highest-performing (and typically most expensive)storage devices, to a lowest storage class of poorest-performing (andtypically least expensive) storage devices. There may be more storagedevices per storage class in lower storage classes than in higherstorage classes, but this is not required. The storage devices may beheterogeneous; that is, there may be different vendors and/or models ofstorage devices in the different storage classes, and there may bedifferent vendors or models of storage devices within a storage class.In other words, storage devices may be grouped in a storage class and/ordivided into different storage classes according to performancecharacteristics rather than make and model.

In one embodiment of the multi-device implementation, the file systemsoftware 142 may address each device as a range of blocks (e.g. from0-n). In one embodiment, the multi-class file system may address aparticular block in the file system as a device:block number, oralternatively device:extent, if extents are used. The device:blocknumber may be associated with a path or directory in the file systemmetadata. The multi-class storage mechanism 144 tracks which storagedevices are assigned to which storage classes. If the multi-class filesystem 144 moves a file or portion of a file to another storage deviceon another storage class, the file or portion of a file remains activein the file system, and the metadata is modified to reflect the move.The device:block number is changed, but the path or directoryinformation remains the same. Thus, the move is transparent to anapplication that accesses the file.

A multi-class file system 160 based on a multi-device feature of a FileSystem may be expanded. For example, storage devices may be added to thestorage classes and/or additional storage classes may be added using theFile System functionality 146, and the multi-class storage mechanism 144may store or migrate files or portions of files to the new storagedevice(s) and/or storage class(es) automatically and transparently tothe application(s) that access the multi-class file system 160. Asanother example, storage allocated to a storage class on a storagedevice within the storage class may be expanded to increase the size ofthe storage class.

In another embodiment, the partitioning technique used to implement amulti-class file system may be multiple independent file systems under avirtualization layer, as illustrated in FIG. 4. File system software 142may include, but is not limited to, an embodiment of the multi-classstorage mechanism 204, software that provides “traditional” File Systemfunctionality 206, and a virtualization layer 208. File Systemfunctionality 206 may be provided by any of various File Systemimplementations including, but not limited to, disk-based file systemssuch as Veritas' VxFS and UNIX File System (UFS), and networked filesystem such as Network File System (NFS) and AT&T's Remote File Share(RFS).

File System functionality 206 may implement a plurality of separate filesystems (e.g., file systems 212A, 212B, and 212C). Each file system 212may be implemented to represent a class in a multi-class file system210. For example, file system 212A may be a highest storage class, filesystem 212B may be a middle storage class, and file system 212C may be alowest storage class. Each file system 212 may be implemented on aseparate storage device, or alternatively one or more of the filesystems 212 may logically extend across two or more storage devices.Each file system 212 may have its own file system metadata.Virtualization layer 208 may be used to map these file systems 212 to asingle file system and present the single file system to one or moreapplication(s). Virtualization layer 208 maintains some form of metadatafor the single file system that maps data paths as used byapplication(s) to data locations in the file systems 212.

Multi-class storage mechanism 204 maintains and implements theuser-defined policies of the multi-class file system 210. Multi-classfile system 210 may initially assign files or portions of files tostorage classes (i.e., file systems) based on the policies, and maymonitor file usage and migrate files or portions of files up or down inthe storage classes according to the policies (in this example, thethree storage classes represented by file system 212A, file system 212B,and file system 212C). To assign or migrate a file or a portion of afile, multi-class file system 210 may interact with File Systemfunctionality 206 to actually move the file from one file system 212 toanother (including modifying the metadata on the affected file systems212), and with virtualization layer 208 to modify the metadata for thesingle file system presented to the application(s) to reflect the move.Alternatively, File System functionality 206 may interact withvirtualization layer 208 to modify the metadata for the single filesystem presented to the application(s) to reflect the move. In themetadata for the single file system presented to the application(s), thepath information is not modified when a file or portion of a file ismigrated. Further, the file or portion of a file remains within thesingle file system presented to the applications(s) by thevirtualization layer 208, and within the file systems 212, and is notmoved offline. Thus, the multi-class file system 210, and migration ofdata within the multi-class file system 210, remains transparent to theapplication(s). All the application(s) see, or need to see, is thesingle file system presented by the virtualization layer 208.

In yet another embodiment, the partitioning technique used to implementa multi-class file system may be block ranges with differentcharacteristics. The block ranges may be implemented by either thesoftware that provides file system functionality (e.g., Veritas' VxFS)or in the underlying volume manager or disk subsystem(s). In thisembodiment, sets of characteristics may be assigned to different rangesof blocks. In the multi-device implementation, the file system softwaremay address each device as a block range (e.g. from 0-n), and addressesone particular block in the file system as device:block number. In theblock range implementation, the device may be made the high-order bitsof the block number, and the concatenated block numbers, separated intoranges representing storage classes, may be used to implement themulti-class file system. Thus, a block range may extend across one ormore devices. In this implementation, one virtual device may bepresented to the application with distinctly different characteristicsassociated with block ranges of the virtual device, with the file systemsoftware understanding what block ranges of the file system have thosecharacteristics.

The multi-class storage mechanism maintains and implements theuser-defined policies of a multi-class file system implemented usingblock ranges as storage classes. The multi-class file system mayinitially assign files or portions of files to storage classes (blockranges) based on the policies, and may monitor file usage and migratefiles or portions of files up or down in the storage classes accordingto the policies. To assign or migrate a file or a portion of a file, themulti-class file system may interact with the software that providesFile System functionality to actually move the file from one block rangeto another (including modifying the metadata to reflect the move). Inthe metadata, the path information is not modified when a file orportion of a file is migrated. Further, the file or portion of a fileremains within the file system presented to the applications(s), and isnot moved offline. Thus, the multi-class file system, and migration ofdata within the multi-class file system, remains transparent to theapplication(s).

Note that other techniques to partition storage devices into storageclasses than those described herein may be used in other embodiments.Embodiments of the multi-class storage mechanism may functiontransparently to the particular underlying partitioning technique usedto organize storage devices or file systems into a multi-class filesystem.

FIG. 5 illustrates a multi-class file system in a network storageenvironment according to one embodiment. Host system 250 may be any ofvarious types of devices, including, but not limited to, a personalcomputer system, desktop computer, laptop or notebook computer,mainframe computer system, workstation, network computer, or othersuitable device. Host system 250 may include at least one processor 252.The processor 252 may be coupled to a memory 254. Memory 254 isrepresentative of various types of possible memory media, also referredto as “computer readable media.” Hard disk storage, floppy disk storage,removable disk storage, flash memory and random access memory (RAM) areexamples of memory media. The terms “memory” and “memory medium” mayinclude an installation medium, e.g., a CD-ROM or floppy disk, acomputer system memory such as DRAM, SRAM, EDO RAM, SDRAM, DDR SDRAM,Rambus RAM, etc., or a non-volatile memory such as a magnetic media,e.g., a hard drive or optical storage. The memory medium may includeother types of memory as well, or combinations thereof. Host system 250may couple, for example over a wired or wireless network or networks, toone or more other devices, e.g. client system 290, via one or more wiredor wireless network interfaces. Host system 250 may couple over anetwork 262 to two or more storage devices, in this example storagedevices 264, 266, 268, 270 and 272. Network 262 may, for example, be aStorage Area Network (SAN), a LAN with Network-Attached Storage (NAS),or any network capable of coupling storage devices to a host system 250.The storage devices may include any of one or more types of storagedevices including, but not limited to, storage systems such as RAID(Redundant Array of Independent Disks) systems, disk arrays, JBODs (Justa Bunch Of Disks, used to refer to disk cabinets that do not have abuilt-in RAID controller), tape devices, and optical storage devices.

Host system 250 may include, in memory 254, file system software 256.File system software 256 may include, but is not limited to, anembodiment of the multi-class storage mechanism 260 and software thatprovides “traditional” File System functionality 258. File Systemfunctionality 258 may be provided by any of various File Systemimplementations including, but not limited to, disk-based file systemssuch as Veritas' VxFS and UNIX File System (UFS), and networked filesystem such as Network File System (NFS) and AT&T's Remote File Share(RFS). In one embodiment, file system software 256 may also include avirtualization layer (not shown) that may be used to map file systems onthe storage devices to a single file system and to present the singlefile system to one or more application(s).

File System functionality 258 of file system software 256 may implement,on one or more of the storage devices, a multi-class file system 280 (inthis example, on storage devices 268, 270, and 272). One of the methodsfor implementing a multi-class file system previously described may beused. In one embodiment, the software that provides the File Systemfunctionality 258 may be used to implement the multi-class file system280, while the multi-class storage mechanism 260 manages the policiesand directs assignment and migration of data within the multi-class filesystem 280 according to the user-defined policies.

The multi-class file system 280 may include one or more physical and/orlogical storage devices organized in a hierarchy of two or more storageclasses according to one or more characteristics of the storage devicesand/or data requirements of the user(s) of the file system. For example,the multi-class file system 280 may have three classes—a first storageclass on storage device 268, a second storage class on storage device270, and a third storage class on storage device 272. As anotherexample, storage device 268 may be a first storage class, and storagedevices 270 and 272 may both be included in a second storage class. Themulti-class file system 280 is made available to the application(s) 292on client system 290 as one file system. The structure of themulti-class file system, and assignment and migration of data within themulti-class file system, are transparent to the application(s) 292.

In one embodiment, each storage class of the multi-class file system 280may have different characteristics such as cost and/or geometry. In oneembodiment, the storage classes may form a hierarchy, with the mostexpensive and typically fastest storage devices in the top or firststorage class, and the least expensive (and typically slower) storagedevices in the bottom or last storage class.

FIG. 6 is a flowchart of a method for assigning and migrating files orportions of files in a multi-class file system according to oneembodiment. As indicated at 300, a multi-class file system including ahierarchy of storage classes may be implemented on one or more storagedevices. In one embodiment, a multi-device feature of file systemsoftware (e.g. Veritas' VxFS) may be used to implement the multi-classfile system. Different architectures for the underlying structure of themulti-class file system were previously described. In one embodiment,each storage class includes one or more storage devices assigned to thestorage class according to one or more characteristics of the storageclass, e.g. performance characteristics. In one embodiment, the storageclasses may be ordered in the hierarchy of storage classes according toperformance characteristics from a highest storage class including oneor more high-performance storage devices to a lowest storage classcomprising one or more low-performance storage devices.

In one embodiment, a multi-class storage mechanism of the file systemsoftware may initially place files in the storage classes in thehierarchy of storage classes according to a user-defined policy forassigning new files to an initial storage class in the hierarchy ofstorage classes.

As indicated at 302, the multi-class storage mechanism of the filesystem software may monitor access of files stored in the hierarchy ofstorage classes to generate access information for the files. Asindicated at 304, the multi-class storage mechanism software may applythe access information and other information about the file (e.g.ownership of the file, business value or importance of the file, etc.)to a set of user-defined policies for initially assigning and migratingfiles in the hierarchy of storage classes.

As indicated at 306, the multi-class storage mechanism of the filesystem software may migrate one or more of the files to differentstorage classes in the hierarchy of storage classes according to the setof user-defined policies when applied to the access or other fileinformation. In one embodiment, less-frequently-accessed files may bemigrated to lower storage classes including lower-performing storagedevices, and more-frequently-accessed files may be migrated to higherstorage classes including higher-performing storage devices. In oneembodiment, files migrated to one or more storage classes in thehierarchy of storage classes, for example to the lowest class, may becompressed. In one embodiment, the multi-class storage mechanismsoftware may modify file system metadata when a file is assigned ormigrated to indicate the storage class for the file. In embodiments,migrating files to different storage classes may be performedtransparently to an application that accesses the files in the hierarchyof storage classes. Note that files migrated to a lowest storage classremain online in the file system, i.e. the data is still present in thefile system and is not moved offline.

In one embodiment, the multi-class storage mechanism software may assignor migrate a portion of a file (e.g. a range of blocks or sectors) toone storage class in the hierarchy of storage classes while assigning orleaving the remainder of the file on a different storage class accordingto the user-defined policies for assigning or migrating files.

In one embodiment, a new storage class may be added to the hierarchy ofstorage classes, and the multi-class storage mechanism software maytransparently migrate one or more of the files or portions of the filesfrom one or more others of the storage classes to the new storage classaccording to the policies. Storage classes may be increased or decreasedin capacity if necessary or desired. In one embodiment, one or morestorage devices may be added to a storage class and data may beautomatically and transparently moved to the new storage deviceaccording to the policies. In one embodiment, a storage device within astorage class may be grown to increase the capacity of the storageclass. If necessary or desired, free disk space on a storage device maybe added to the storage class to grow the storage class. Data may beautomatically and transparently moved to the newly allocated disk spaceon the storage device according to the policies.

FIGS. 7A through 7E illustrate the migration process in an exemplarymulti-class file system with three storage classes according to oneembodiment. In this example, file system software may include amulti-class storage mechanism as previously described. The file systemsoftware may, as previously described, implement a multi-class filesystem 240 including a hierarchy of storage classes on one or morestorage devices; in this example there are three storage classes 220,222, and 224, each of which may include one or more storage devices.Note that other multi-class file systems may have different numbers ofstorage classes. Also note that the term “data” as used in this exampleindicates a file or a portion of a file (e.g., a range of blocks or oneor more extents).

The multi-class storage mechanism may maintain a set of policies fordata assignment and migration within the multi-class file system, maymonitor data accesses within the multi-class file system, and may causedata being monitored to be assigned to storage classes and/or migratedto different storage classes within the multi-class file systemaccording to the set of policies. In this example, first storage class220 may be mirrored on one or more high speed disks, and activelywritten data may be assigned or migrated there according to thepolicies. Data that have not been written to for some period (e.g. inthe last month) are migrated to the second storage class 222 accordingto the policies, and data that have not been written to for some period(e.g. in the last quarter) are migrated to the third storage class 224according to the policies. The second storage class 222 and thirdstorage class 224 may each include one or more storage devices, whichmay be less expensive and slower than the storage device(s) of the firststorage class (e.g. RAID-5 array(s)); the third storage class 224 mayinclude one or more less expensive, and slower, storage devices than thestorage device(s) of the second storage class 222. If data in the secondstorage class 222 or third storage class 224 is written to, it may bemigrated back to the first storage class 220 according to the policies.

FIGS. 8A through 8D illustrate an exemplary metadata structure for data230 of FIGS. 7A-7D that may be modified by the file system softwareduring assignment and migration of data 230 in multi-class file system240 according to one embodiment. As the migration process is describedusing FIGS. 7A-7E, FIGS. 8A-8D will be referenced to indicatemodifications in the metadata to reflect the migration(s).

As illustrated in FIG. 7A, data 230 may initially be written to firststorage class 220 of multi-class file system 240. Data 230 may be a fileor a portion of a file (e.g. one or more records of a database file, arange of blocks, or one or more extents of a file). The data may havebeen generated by some application using the multi-class file system240. From the application's perspective, the data 230 was created atsome logical location in the file system as seen by the applicationindicated to the application by a path. For example, a path to data 230may look something like “MyDrive/MyFolder/MyFile” Note that this exampleis exemplary and is not intended to be limiting; other methods forindicating a path to data in a file system are possible.

FIG. 8A illustrates metadata for data 230 when assigned to first storageclass 220. When data 230 is assigned to the first storage class 220, thedata 230 is written to a block range on a storage device in the firststorage class 220. In this illustration, the path as seen by theapplication is shown on the left, and the location information for thedata 230 in the multi-class file system 240 as used by the file systemsoftware is shown on the right. The location information for the data230 in the multi-class file system 240 indicates that the data 230 islocated at “First Storage Class:Device:Block Number”. Note that thisexample is exemplary and is not intended to be limiting; other methodsfor storing location information for data in a multi-class file systemare possible. For example, an alternative is to simply indicate thedevice and block number, as the storage class may be determined from thedevice identifier. Another alternative is to concatenate the deviceidentifier and block number (as is done in embodiments where theunderlying implementation of the multi-class file system uses blockranges as storage classes, as previously described). Also note that themetadata may include other information not shown, such as the number ofblocks the data 230 occupies starting at the indicated block number.

As illustrated in FIG. 7B, at some point migration 232 of data 230 fromfirst storage class 220 to second storage class 222 may occur. Forexample, the multi-class storage mechanism may monitor (read and/orwrite) access of data on first storage class 220, and periodicallymigrate data that has not been accessed for a period of time (e.g. onemonth) to second storage class 222 according to the policies. In thisexample, after data 230 is initially assigned to first storage class220, the multi-class storage mechanism may monitor access of data 230and, if the data has not been accessed for a period of time as specifiedby the policies, may migrate data 230 to second storage class 222. Themigration 232 does not move data 230 offline; data 230 remains onlinewithin the file system. Note that the policies may include othercriteria than simply access criteria to determine when and if data is tobe migrated. For example, the policy may indicate that data 230 iscritical to business operations, or is owned by a particular user orapplication, and is therefore not to be migrated. As another example,the policy may specify that all actively accessed data is to be storedon first storage class 220 and migrated to second storage class 222after a month of inactivity except data that belongs to a particularnon-critical application, which is to be migrated to second storageclass 222 after one week of inactivity. Using the policies, othercriteria, including business-related criteria, than simply data accesscriteria may be applied by the multi-class storage mechanism in makingassignment and migration decisions.

FIG. 8B illustrates the metadata for data 230 after migration 232 tosecond storage class 222. Note that the location information for thedata 230 in the multi-class file system 240 used by the file systemsoftware now indicates that data 230 is stored starting at a blocknumber on a device of the second storage class 222. Also note that thepath as seen by the application has not changed; from the application'sperspective, migration 232 is transparent. The migration 232 does notmove data 230 offline; data 230 remains online within the file system.

As illustrated in FIG. 7C, at some point migration 234 of data 230 fromsecond storage class 222 to the third storage class 224 may occur. Forexample, the multi-class storage mechanism may monitor (read and/orwrite) access of data on the second storage class 222, and periodicallymigrate data that has not been accessed for a period of time (e.g. threemonths) to third storage class 224 according to the policies. In thisexample, after data 230 has migrated to second storage class 222, themulti-class storage mechanism may monitor access of data 230 and, if thedata has not been accessed for a period of time as specified by thepolicies, may migrate data 230 to third storage class 224. The migration234 does not move data 230 offline; data 230 remains online within thefile system. Note that the policies may include other criteria thansimply access criteria to determine when and if data is to be migrated.Using the policies, other criteria, including business-related criteria,than simply data access criteria may be applied by the multi-classstorage mechanism in making assignment and migration decisions.

FIG. 8C illustrates the metadata for data 230 after migration 234 tothird storage class 224. Note that the location information for the data230 in the multi-class file system 240 used by the file system softwarenow indicates that data 230 is stored starting at a block number on adevice of the third storage class 224. Also note that the path as seenby the application has not changed; from the application's perspective,migration 234 is transparent. The migration 234 does not move data 230offline; data 230 remains online within the file system.

While FIGS. 7A-7C show data migrating from a higher storage class downto a next lower storage class, note that data may be migrated from anystorage class to any other storage class in the hierarchy of storageclasses according to the policies. For example, when migrating data downin the hierarchy of storage classes, some data may be migrated from thehighest storage class to the next lower storage class, and other datamay be migrated directly to the lowest storage class according to thepolicies.

FIGS. 7D and 7E illustrate an exemplary method of handling of an accessof data 230 on third storage class 224 according to one embodiment. Atsome point after migration 234 of data 230 to third storage class 224,an application may generate an access request (e.g. a read or writerequest) for the data 230. In FIG. 7D, the file system software mayprovide data 230 to the application to satisfy the access request, asindicated at 236. The policies may indicate to the multi-class storagemechanism that data stored on third storage class 224 is to be migratedto first storage class 220 if the data is accessed. In FIG. 7E, themulti-class storage mechanism may migrate 238 data 230 from thirdstorage class 224 to first storage class 220 in response to the accessof data 230 by the application if so indicated by the policies. Themigration 238 does not have to restore data 230 from offline storageback into the file system; data 230 was online within the file systemprior to the migration 238.

Note that, alternatively, data 230 may be migrated 238 to first storageclass 220 prior to providing the data 230 to the application asindicated at 236. Also note that the policies used by the multi-classstorage mechanism to make migration decisions may indicate some otheraction, which may be determined using other criteria than simply dataaccess criteria, than migration to first storage class 220 upon accessof data 230 is to be performed. For example, the policies may indicatethat, if the access is a write access, data 230 is to be migrated tofirst storage class 220, but if the access is a read access, data 230 isto remain on third storage class 224. As another example, the policiesmay indicate that an access of data 230 by a particular application (oruser) does not require data 230 to be migrated, but access by anotherapplication (or user) does require data 230 to be migrated.

While FIGS. 7D and 7E show data migrating up from a lowest storage classto a the highest storage class, note that data may be migrated from anystorage class to any other storage class in the hierarchy of storageclasses according to the policies. For example, when migrating data upin the hierarchy of storage classes, some data may be migrated from thelowest storage class to the next higher storage class, and other datamay be migrated directly to the highest storage class according to thepolicies.

FIG. 8D illustrates the metadata for data 230 after migration 238 tofirst storage class 220 from third storage class 224. Note that thelocation information for the data 230 in the multi-class file system 240used by the file system software now indicates that data 230 is storedstarting at a block number on a device of the first storage class 220.Note that the device is not necessarily the same device as in themetadata of FIG. 8A, and the block number is not necessarily the sameblock number as in the metadata of FIG. 8A. Also note that the path asseen by the application has not changed; from the application'sperspective, migration 238 is transparent.

One embodiment may include one or more storage classes of writeablestorage and one or more storage classes that are not writeable(read-only). Note that, in one embodiment, storage devices within astorage class may be segmented into subclasses of writeable andread-only storage devices. Files may be migrated down to a read-onlystorage class, but the read-only storage class is only written to duringmigration. Thus, there may be one or more read-only storage classes in amulti-class file system. To write to one of the files in a read-onlystorage class, the file may first be migrated up to a writeable storageclass to be written to. The file may remain in the higher storage classfor a period until the multi-class storage mechanism determines that thefile may be migrated back to the read-only storage class, oralternatively may be immediately migrated back to the read-only storageclass, according to the policies.

In one embodiment, it may not be necessary to migrate an entire filebecause the file mapping in the file system may be implemented so thatthe blocks or sectors of the file that have not been modified may remainon a read-only storage class and the blocks or sectors that are to bemodified are migrated to a writeable storage class according to thepolicies. Thus, parts of files as well as entire files may be migrated,and different blocks or sectors of a file may concurrently reside ondifferent storage classes. To write to a file on a read-only storageclass, blocks, sectors or records (in a database) that need to bemodified may be migrated to a writeable storage class while the rest ofthe file stays on the read-only storage class.

The ability to migrate a portion of a file may be an important featurefor databases, as an example. Once completed, a transaction in adatabase is typically not modified again. All transactions that have notbeen completed yet may reside in a writeable storage class.Periodically, completed transactions may be migrated to a read-onlystorage class (the database application may still need to access thetransactions, so keeping them available is preferable). Inactive orcompleted records in a database may be migrated down to a read-onlystorage class while records that are still active are maintained in awriteable storage class according to the policies.

In one embodiment, a multi-class file system may include one or morelevels of general storage classes, with one or more storage classes ineach general storage class. A general storage class, like a storageclass, may be based, for example, on cost/performance characteristics ofthe storage devices. Two or more storage classes in a general storageclass may be considered at the same level in the hierarchy of storageclasses. Data may be stored on or migrated to different classes ofstorage within a more general storage class according to the policies.For example, a general read-only storage class may include a storageclass for actively accessed read-only data and a storage class for non-(or rarely-) accessed read-only data, the latter of which may becompressed. As another example, a general writeable storage class mayinclude two or more classes based on criteria such as frequency ofaccess. Note that other criteria than frequency of access may be appliedto segment data within storage classes in a general storage class. As anexample, importance of the data may be used as a criterion; some datawithin a general storage class may be deemed more important or criticalthan other data, and thus may be segmented from the less important databy storing the different data on different storage classes within thegeneral storage class.

Using Compression in a Multi-Class File System

Embodiments may use compression to achieve further data storageefficiency in a multi-class file system. In a multi-class file system,there may be one or more classes of storage that contain files or blocksthat are not actively written. By compressing these files (or blocks), asignificant amount of disk space may be saved. One embodiment maycompress all files or ranges of blocks migrated to a lower or loweststorage class. Rather than compressing all the files or blocks in astorage class, another embodiment may gather usage statistics for filesor blocks in a storage class or storage classes and compress files (or,alternatively, ranges of blocks) in the storage class that are lessfrequently accessed.

Since it is known that the files or block ranges to be compressed arenot being actively written, the file system may choose a compressionalgorithm that provides better space savings at the expense of not beingable to do efficient in-place updates. In one embodiment, data in older(lower) storage classes may be compressed.

One embodiment may keep statistics on file usage and compress less-used(below some usage threshold) files and/or ranges of blocks. This maysave disk space without the processing cost of decompressing when a fileis read. One embodiment may perform the compression when migrating to alower storage class for inactive files, as the file system has had timeto gather statistics on the file access patterns. This may be used toimplement only changing data on the lower storage classes at fixedtimes.

In one embodiment, when migrating data in files or portions of files toa lower storage class, the data may be compressed. One of thecharacteristics of files that may be examined by a policy-drivenmulti-class storage mechanism is files that have not been written to forsome period. A typical problem with compression in file systems is that,when a file is to be compressed, it is not known in advance exactly howmuch compression will be achieved. A file is essentially random-access;if one byte or block in the middle of a compressed file is changed(typically by decompressing the file, changing the byte, andrecompressing the file), the change to the byte or block may potentiallychange the compression size. For an uncompressed file, rewriting oneblock does not change the file size. If a compressed file is changed,recompressing the file might not leave enough space for the file, as therecompressed file may be larger than the original compressed files.

In one embodiment, files that have not been modified for a period (e.g.a month) may be migrated to a lower storage class. Files from thatstorage class may be migrated to an even lower storage class if thefiles are not modified for another, longer period (e.g. six months). Thefiles may be compressed when moving to the even lower storage classbased on the polices with confidence that the files are not likely to beuncompressed and modified (the files have not been modified for thelonger period, so are not likely to be modified), achieving further costsavings. Using a policy such as this, processing resources are notwasted compressing files that are likely to be uncompressed in thefuture, and further the problem of not having space for recompressedfiles described above may be avoided.

Embodiments may implement compression and decompression as part ofmigration from one storage class to another. In one embodiment,different compression algorithms may be used depending on the type ofdata being migrated. One embodiment may use a compression algorithm oralgorithms that decompresses very fast, with a high a level ofcompression. Compression may not need to be particularly fast sincecompression is performed on files that are not likely to be accessed;decompression is performed when access to the file is desired orrequired, so decompression may have time constraints that compressiondoes not.

Backup Mechanism for a Multi-Class File System

Embodiments of the multi-class storage mechanism and the multi-classfile system may be used in combination with a backup mechanism to reduceone or more of, but not limited to, total media used, the number of tapedrives (or other offline storage devices) needed, the backup window, andrestore time.

Embodiments of the multi-class storage mechanism used in combinationwith a backup mechanism may provide the ability to schedule backups tosave backup media storage space as well as backup (and restore) time. Inone embodiment, a full backup of all the storage classes, including oneor more higher, writeable storage classes and one or more lower,read-only storage classes, may be performed periodically, andincremental backups may be performed on the writeable storage class(es)between the full backups of the writeable storage classes. For example,data in read-only storage class may be periodically backed up in animage-based full backup while the system remains online. Periodically,incremental backups of the writeable storage class(es) may be performed.Thus, the read-only data is segregated into its own full backup. Duringrestore, the writeable storage classes may be restored first, theapplications may be started up, and then the inactive data (read-only)data may be restored while the system is up. Thus, both the backupwindow and the restore window may be shorter than in conventional backupand restore mechanisms. Further, in one embodiment, there may be lessoverall work when doing backups and restores, as there is no need to doa split mirror of storage devices in read-only storage classes.

By organizing storage into a hierarchy of storage classes including oneor more writeable storage classes and one or more read-only storageclasses, embodiments may reduce backup media (e.g. tape) costs, as wellas scanning of file system costs, since the backup and restore does nothave to walk through the entire file tree.

FIGS. 9-12 illustrate means for migrating aged data and performingbackups of the storage classes in a hierarchy of storage classes of amulti-class file system. FIG. 9 illustrates a backup mechanism for anexemplary multi-class file system on a host system in a network storageenvironment according to one embodiment. Host system 370 may be any ofvarious types of devices, including, but not limited to, a personalcomputer system, desktop computer, laptop or notebook computer,mainframe computer system, workstation, network computer, or othersuitable device. Host system 370 may include at least one processor 372.The processor 372 may be coupled to a memory 374. Memory 374 isrepresentative of various types of possible memory media, also referredto as “computer readable media.” Hard disk storage, floppy disk storage,removable disk storage, flash memory and random access memory (RAM) areexamples of memory media. The terms “memory” and “memory medium” mayinclude an installation medium, e.g., a CD-ROM or floppy disk, acomputer system memory such as DRAM, SRAM, EDO RAM, SDRAM, DDR SDRAM,Rambus RAM, etc., or a non-volatile memory such as a magnetic media,e.g., a hard drive or optical storage. The memory medium may includeother types of memory as well, or combinations thereof. Host system 370may couple, for example over a wired or wireless network or networks, toone or more other devices, e.g. client systems, storage devices andbackup devices, via one or more wired or wireless network interfaces.The network may, for example, be a Storage Area Network (SAN), a LANwith Network-Attached Storage (NAS), or any network capable of couplingdevices to a host system 370. The storage devices may include any of oneor more types of storage devices including, but not limited to, storagesystems such as RAID (Redundant Array of Independent Disks) systems,disk arrays, JBODs (Just a Bunch Of Disks, used to refer to diskcabinets that do not have a built-in RAID controller), tape devices, andoptical storage devices.

Host system 370 may include, in memory 374, file system software 352.File system software 352 may include, but is not limited to, anembodiment of the multi-class storage mechanism and software thatprovides “traditional” File System functionality. File Systemfunctionality may be provided by any of various File System softwareimplementations including, but not limited to, Veritas' VxFS software orNFS software. In one embodiment, the software that provides the FileSystem functionality may be used to implement the multi-class filesystem 380 including a hierarchy of storage classes 382, while themulti-class storage mechanism of the file system software manages thepolicies and directs assignment and migration of data within themulti-class file system 380 according to the user-defined policies.

Host system 370 may also include, in memory 374, a backup mechanism 350that may interact with file system software 352 and with the multi-classfile system to perform backups of the storage classes 382 of multi-classfile system 380. Backup mechanism 350 may periodically perform fullbackups of each of the storage classes 382. In one embodiment, the fullbackups may be image-based backups of the storage devices in the storageclasses 382. In one embodiment, the lower storage classes are backed upless frequently than the higher storage classes.

In one embodiment, one or more of the lower storage classes may beread-only storage classes, and data stored on a higher, writeablestorage class that has not been written for a period (e.g. one month)may be periodically migrated (e.g. once a month) to a lower, read-onlystorage class. Data stored on read-only storage class that has not beenaccessed for a period (e.g. three months) may be periodically migrated(e.g. every three months) to a lower read-only storage class. In oneembodiment, data on a read-only storage class that needs to be writtento may first be migrated to a higher, writeable storage class. Thus, thebackup mechanism 350 may periodically perform image-based full backupsof the lower, read-only storage classes at longer intervals than backupsof the higher, writeable storage classes. In one embodiment, themigration schedule for a storage class may coincide with the backupschedule of the storage class.

For example, in a multi-class file system containing a writeable storageclass and two read-only storage classes, the writeable storage class mayhave a periodic (e.g. weekly) image-based full backup performed, alongwith incremental backups (e.g. performed daily). Periodically, e.g.monthly, data that have been inactive on the writeable storage class forat least that period (e.g. one month), may be migrated to the firstread-only storage class. After the periodic (e.g. monthly) migration ofinactive data from the writeable storage class to the first read-onlystorage class, a monthly image-based full backup of the first read-onlystorage class may be performed. At a longer period (e.g. every threemonths), data that have been inactive on the first read-only storageclass for at least the longer period (e.g. three months) may be migratedto the second read-only storage class. After this migration, animage-based backup of the second read-only storage class may beperformed. Note that these migrations leave the data active within thefile system and do not move the data offline.

In one embodiment, if data on one of the read-only storage classes needsto be modified, the data are first migrated to the writeable storageclass. This ensures that the read-only storage classes include only datathat is not actively written, and that the modified data is on thewriteable storage class where it will be more frequently backed up. Inone embodiment, when data are migrated from one of the read-only storageclasses to the writeable storage class, or if data on one of theread-only storage classes is deleted (e.g. if the file containing thedata is deleted), the data blocks on the read-only storage class thatcontain the data are left untouched; the migration or deletion isindicated by changing the metadata to indicate the new location (ordeletion) of the data. Thus, until the next periodic migration of datato the read-only storage class (which may potentially overwrite the datablocks), deleted data on the read-only storage class may be recovered bysimply modifying the metadata rather than restoring the data fromoffline storage. In one embodiment, file block information from the mostrecent image backup may be used to recover and restore the block map.

Using this method of backing up a multi-class file system may result inreduced requirements for backup media (e.g. tapes) and potentiallybackup devices (e.g. tape drives). In one embodiment, files migrated toone or more lower storage classes 382 may be compressed, resulting infurther savings of backup media.

Note that, while this example shows backup mechanism 350 and file systemsoftware 352 in memory 374 on the same host system 370, backup mechanism350 and file system software 352 may reside on different systems. Forexample, backup mechanism 350 may reside on a different host systemcoupled to host system 370.

FIG. 10 illustrates a backup mechanism for an exemplary multi-class filesystem with three storage classes according to one embodiment. In oneembodiment, a multi-class file system 380 may be set up with a hierarchyof storage classes. The storage classes may be created using any of themechanisms previously described. In this example, file system software352 may include a multi-class storage mechanism as previously described.File System functionality of file system software 352 may, as previouslydescribed, implement a multi-class file system 380 on one or morestorage devices; in this example, there are three storage classes.Backup mechanism 350 may interact with the multi-class file system 380to perform backups of the three storage classes in this exampleaccording to the following description. Note that a similar method maybe applied in multi-class file systems with different numbers of storageclasses. Also note that, while the term “file” is used in this example,the description also applies to portions of files (e.g. ranges of blocksor extents).

In this example, a first class of data storage (first storage class 354)is mirrored on the highest speed disks and contains all the dataactively being written. The second storage class 356 is a read-onlystorage class that contains all data that have not been written in thelast month, and the third storage class 358 is a read-only storage classthat contains all data that have not been written in the last quarter.The second and third storage classes may include less expensive storagedevices than the first storage class 354. In one embodiment, each classof storage may be monitored for free space and automatically grown asneeded. In one embodiment, if data in a read-only storage class needs tobe written to, it is first migrated to a writeable storage class. In oneembodiment, data may be compressed when migrated to the read-onlystorage classes.

If a multi-device file system is being used for the multi-class filesystem 380, then, in one embodiment, all the metadata may be stored onthe first storage class 354 and the block mappings for files on thesecond and third classes may be kept stable. To keep the block mappingsstable, if files are removed or migrated back to the first storage class354, then the blocks are simply left unused in the second or thirdstorage class and not written over. This allows an image-based backup ofthe first storage class 354 to be performed, and for the image-basedbackup of the first storage class 354 to be used in conjunction with anearlier image-based backup of the second and third storage classeswithout worrying about block aliasing. In one embodiment, the blockmappings in the second and third storage classes may be stablymaintained, and a write lock mechanism may be used on the lower (in thisexample, second and third) storage classes, along with a timestamp thatindicates the last time the write lock was enabled. Using a write lockmechanism may allow a backup mechanism to verify the fidelity of thosestorage classes.

As an example, suppose that there are three storage classes, and thatall three storage classes may be modified. Metadata is on a firststorage class, and is backed up every week. In week 1, a file thatresides on the third class is located on blocks 1-10. If the file iswritten in week 2, the file may be moved to the second storage class andlocated on blocks 101-110. If the image-based backup of week 2 iscombined with the image-based backup of the third storage class fromweek 1, the file would point to the wrong blocks. This is the problem ofblock aliasing, or mistracking the identity of block(s). To ensure thatstorage classes backed up at the same time are static in terms of blockmapping, in one embodiment, between the times that storage classes arebeing migrated to, file reorganization, extensions, or modifications arenot allowed to occur within the storage classes. It may be permitted toremove a file; however, writing to the space vacated by the file is notallowed until the next time a file is migrated down to the device. Inthis embodiment, the images of the different storage classes may bebacked up to get a reliable image of the file system. Thus, in oneembodiment, when data is migrated down (or up) to a storage class, a newimage backup of the storage class(es) may be generated. In thisembodiment, by making one or more lower storage classes read-only andkeeping the block mappings static if possible, it may be known that theblocks on the image backups match what the metadata says they are.

One embodiment may include one or more storage classes of writeablestorage and one or more storage classes that are not writeable(read-only). Note that, in one embodiment, storage devices within astorage class may be segmented into subclasses of writeable andread-only storage devices. Files may be migrated down to a read-onlystorage class, but the read-only storage class is only written to duringmigration. To write to one of the files in a read-only storage class,the file may first be migrated up to a writeable storage class to bewritten to. The file may remain in the higher storage class for a perioduntil a the multi-class storage mechanism implementing the policiesdetermines that the file is to be migrated back to the read-only storageclass, or alternatively may be immediately migrated back to theread-only storage class according to the policies.

In one embodiment, it may not be necessary to migrate an entire filebecause the file mapping in the file system may be implemented such thatthe blocks or sectors of the file that have not been modified remain onthe read-only storage class and the blocks or sectors that are to bemodified are migrated to the writeable storage class. Thus, parts offiles as well as entire files may be migrated, and different blocks orsectors of a file may concurrently reside on different storage classes.To write to a file on a read-only storage class, blocks, sectors orrecords (in a database) that need to be modified may be migrated to awriteable storage class while the rest of the file stays on theread-only storage class.

The ability to migrate a portion of a file may be an important featurefor databases, as an example. Once completed, a transaction in adatabase is typically not modified again. All transactions that have notbeen completed yet may reside in a writeable storage class.Periodically, completed transactions may be migrated to a read-onlystorage class. The database application may still need to access thetransactions, so the transaction records are kept online and available(files migrated within the multi-class file system remain online in thesame file system). Inactive or completed records in a database may bemigrated down to a read-only storage class (remaining online in the filesystem) while records that are still active are stored in a writeablestorage class.

Thus, embodiments of the multi-class storage mechanisms may know thatdata on a read-only storage class is not changing. For backups, it maybe known that a large percentage of data (e.g. 80%) has not changed, sothe active data (e.g. 20%) may be backed up more often (e.g. everyweek), and the inactive data (e.g. 80%) may be backed up less often(e.g. every month). Thus, the total amount of storage media, e.g. tape,necessary for backup may be preferably reduced by a considerable amount,as well as reducing backup time.

FIGS. 11A-11D illustrate an exemplary migration and backup schedule foran exemplary multi-class file system according to one embodiment. Inthis example, first storage class 354 may be a writeable storage class,and the second and third storage classes 356 and 358 may be read-onlystorage classes. The read-only storage classes may have write locks forpreventing data to be written to the storage classes (unless the writelock is disabled) and a time stamp for determining the last time theread-only storage classes were written to. Note that this is anexemplary multi-class file system and is not intended to be limiting; amulti-class file system may be configured with one or more writeablestorage classes and one or more read-only storage classes, onlywriteable storage classes, or even only read-only storage classes.

In FIG. 11A, data 386 are initially written to the first storage class354. Periodically (e.g., once a week) a full backup 360 of the firststorage class 354 is performed. The full backup 360 may be an imagebackup. Incremental backups 362 of the first storage class 354 may beperformed between full backups 360, for example every day. Theincremental backups 362 may be file-based backups.

In FIG. 11B, periodically (e.g., once a month), the write lock isdisabled on the second storage class 356, and data 390 in the firststorage class 354 that has not been written to for at least that period(e.g. for at least a month) is migrated from the first storage class 354to the second storage class 356, as indicated at 388. Note that migrateddata 390 remains online in the file system. After migration, the writelock is re-enabled for the second storage class 356. A full backup ofthe second storage class 356 may be generated by the backup mechanismafter the data 390 are migrated. Full backup 364 may be an image backup.In addition, a full backup of the first storage class 354 may beperformed after migration 388. In one embodiment, data 390 may becompressed when migrated to second storage class 356.

In FIG. 11C, periodically, (e.g. once a quarter), the write lock isdisabled on the third storage class 358, and data 394 in the secondstorage class 356 that has not been written to for at least that period(e.g. for at least three months) is migrated from the second storageclass 356 to the third storage class 358, as indicated at 392. Note thatmigrated data 390 remains online in the file system. After migration,the write lock is re-enabled for the third storage class 358. A fullbackup of the third storage class 358 may be generated by the backupmechanism after the data 394 are migrated. Full backup 358 may be animage backup. In addition, a full backup of the first storage class 354may be performed after migration 392. In one embodiment, data 394 may becompressed when migrated to third storage class 358.

With this schedule, the third storage class 358 may only be backed up atlonger intervals, for example every three months. The second storageclass 356 may be backed up at shorter intervals than the third storageclass 358, for example once a month. The backups of the read-onlystorage classes may be performed in conjunction with the migration ofaged data to the read-only storage classes. In one embodiment, the fullbackups of the read-only storage classes may be done online withoutusing split mirrors, since the storage classes are write locked andstable. The first storage class 354 and the metadata may have a fullbackup 360 more often, for example weekly, along with a set ofincremental backups 362 (which may be file-based backups) performed, forexample, daily. In one embodiment, the full backups of all storageclasses may all be image-based, which is typically faster thanfile-based backup. Thus, there is no need for a synthesized full backup.

FIGS. 11A-11C illustrate migrating aged data from a first storage classto a second class, backing up the second storage class, migrating ageddata from the second class to a third storage class, and backing up thethird storage class. In an alternative migration and backup schedule,aged data may first be migrated from a second storage class to a thirdstorage class, and then the third storage class may be backed up. Aftercompletion of the migration to the third storage class, aged data may bemigrated from the first storage class to the second storage class, andthen the second storage class may be backed up. Finally, the firststorage class may be backed up after migrating aged data to the secondstorage class. This migration and backup schedule empties the secondstorage class of aged data before migrating in the new aged data fromthe first storage class. Further, files are not backed up from thesecond storage class and then migrated to the third storage class to bebacked up again. Similarly, files are not backed up from the firststorage class and then migrated to the second storage class to be backedup again. Thus, for this migration and backup schedule, the order ofFIGS. 11A-11C may be reversed.

FIG. 11D illustrates migrating data to be modified from a read-onlystorage class to a writeable storage class. In this example, data 396 onthird storage class 358 may need to be modified. Data 396 is migratedfrom the read-only third storage class 358 to the writeable firststorage class 354 prior to modification, and may then be modified on thewriteable first storage class 354.

In one embodiment, a read-only storage class remains stable betweenperiodic migrations of aged data. When data that needs to be modified ismigrated from a read-only storage class to a writeable storage class,the data blocks in the read-only storage class may not be overwrittenuntil the next migration of aged data to the read-only storage class.The data is copied to the writeable storage class, and the metadata forthe data is modified to indicate the new location for the data inmulti-class file system 380. Similarly, when data is deleted from aread-only storage class, the data blocks in the read-only storage classmay not be overwritten until the next migration of aged data to theread-only storage class. The metadata for the data is modified toindicate the data has been deleted. Thus, if data that was deleted sincethe last periodic migration needs to be restored to a read-only storageclass, instead of restoring the data from a backup of the storage class,a block map may be used (e.g. a block map from a full backup performedafter the last migration of aged data) to restore the metadata toindicate that the data is on the read-only storage class at the blocklocations indicated by the block map from the full backup.

To restore the entire multi-class file system 380, the last full backupof each storage class may be restored, and then the incremental backupsof the first storage class 362 may be applied. In one embodiment, a fullbackup of the first storage class 360 may first be applied, then anynecessary incremental backups of the first storage class 362, followedby the full backups of the second and third storage classes. In oneembodiment, the multi-class file system 380 may be brought back onlineand made available to application(s) after the full and incrementalbackups of the first storage class 354 are restored and before the fullbackups of the other storage classes have been (fully) restored.

Consider as an example, a multi-class file system where 10% of the datais active and stored according to the polices on a first storage class,20% of the data was written in the last quarter and is stored accordingto the policies on a second storage class, and 70% of the data has beenunwritten for the last quarter and is stored according to the policieson a third storage class. In this example, each week, 10% of the datamay be backed up offline, so the backup window may be reduced by afactor of 10. Split mirrors for only 10% of the data are required,reducing the cost of extra disks for backup. In addition, cost topreserve offsite copies of the media is reduced. For 52 weeks of fullbackups, 52 times as much tape as there is disk would be needed. Usingan embodiment of the backup mechanism, 4*.7 plus 12*.2 plus 52*.1, or9.4 times as much tape as disk, may be needed. This saves about 80% oftape media cost and, since the weekly backups are much smaller, theremay be savings on drives as well.

Embodiments of a backup mechanism for multi-class file systems mayenable an optimized restores of data to the multi-class file systems. Todo a restore using the above example, the image backup of the active 10%may be restored to the first storage class. Then, the incrementalbackups (which may be file based) may be applied. At this point, all theactive data has been restored. The application(s) that access the datamay then be started and may access the active data while the second andthird storage classes are being restored. In effect, operations havebeen restarted after restoring only about 10% of the data. In oneembodiment, to handle read requests for data on the second or thirdstorage classes, either the volume manager or the file system softwaremay block the request until the data has been restored. In oneembodiment, a snapshot mechanism of a volume manager (e.g., Veritas'Volume Manager) may be used to do this and, since the restore is imagebased, the restore mechanism may skip ahead to restore requested blocksat the expense of slowing down the completion of the restore slightly.

One embodiment may provide the ability to perform device-oriented ordevice-level backup rather than strictly file-level backups. Data, evenwithin files, that is not being written may be segregated from activelywritten data. An image backup of the devices within the read-onlystorage classes may be performed periodically, e.g. every one, three, orsix months. More active data on storage devices within the writeablestorage classes may be backed up more often, e.g. every month. The mostactive data on storage devices within a highest storage class may bebacked up even more often. Data is segregated by storage class, so inone embodiment image backups of storage devices in different storageclasses may be performed. Image backups tend to be faster thanfile-based or other backups, and only data that is needed for thebackups of the particular storage classes may be backed up in the imagebackups.

In one embodiment of the backup mechanism, incremental backups may backup just the data blocks or records within a file that have been changed,and not the entire file. Unmodified portions of a file remain on deviceswithin the read-only storage classes, while modified portions of thefile are stored on devices within writeable storage classes. The backupmechanism may perform incremental backups of just the writeable storageclasses, and thus may only back up the portions of the file that havebeen modified and not the entire file.

In one embodiment, the storage classes within the multi-device filesystem may themselves be layered, and the backup/restore mechanism maybe applied within a storage class. For example, a read-only storageclass may be broken into levels of actively accessed read-only data andnon- or rarely-accessed read-only data, the latter of which may becompressed. Each of these levels may be stored on different subclassesof storage within the more general read-only storage class. The activelyaccessed read-only data may be backed up more frequently than the non-or rarely-accessed data. This segmentation may also be applied to otherclasses of storage, e.g. a writeable storage class may be segmented intotwo or more subclasses based on criteria such as frequency of access.Note that other criteria than frequency of access may be applied withina storage class to create subclasses or divisions within the storageclass. As an example, importance of the data may be used as a criterion;some data within a storage class may be deemed more important orcritical than other data, and thus may be segmented from the lessimportant data (e.g. by storing the different data on different storagedevices within the storage class) and a backup schedule may be gearedtowards selectively backing up the qualitatively different data based onthe criteria used to segment the data.

Note that embodiments may perform individual file backups and restoresin addition to image backups and restores. For serious or widespreaddamage, the entire multi-class file system may be restored.

FIG. 12 is a flowchart of a method for performing backups of amulti-class file system according to one embodiment. As indicated at400, data may be assigned and migrated in a multi-class file systemincluding a plurality of storage classes. The multi-class file systemmay be implemented on one or more storage devices, e.g. by File Systemfunctionality of the file system software. In one embodiment, amulti-device feature of the file system software (e.g. Veritas' VxFS)may be used to implement the multi-class file system. Differentmechanisms for implementing the underlying hierarchy of the multi-classfile system were previously described. In one embodiment, each storageclass includes one or more storage devices assigned to the storage classaccording to one or more characteristics of the storage class, e.g.performance characteristics. In one embodiment, the storage classes maybe ordered in a hierarchy according to performance characteristics froma highest storage class including one or more high-performance storagedevices to a lowest storage class comprising one or more low-performancestorage devices.

Recently accessed files may be assigned and/or migrated to higherstorage classes and less recently accessed files may be assigned and/ormigrated to lower storage classes in the hierarchy by a multi-classstorage mechanism according to a set of user-defined policies. In oneembodiment, one or more of the higher storage classes may be designatedas writeable storage classes for storing data that have been recentlymodified, and one or more of the lower storage classes may be designatedas read-only storage classes for storing “aged” data that have been lessrecently modified. In one embodiment, file system metadata may bemodified to indicate assignment or migration of the files. In oneembodiment, the file system software may migrate portions of a file todifferent storage classes according to the user-defined policies. Forexample, actively written records of a database file may be stored in ahigher storage class of the multi-class file system, with inactivedatabase records migrated to a lower or lowest storage class. In oneembodiment, files migrated to lower storage classes may be compressed.

As indicated at 402, the file system software may provide access to thedata in the multi-class file system to one or more applications. Theapplications may write new data to one or more of the storage classes,access data on the writeable storage class(es) for read or writeoperations, and access data on the read-only storage classes for readoperations. If an application needs to access data currently stored on aread-only storage class for a write operation, the data are firstmigrated to a writeable storage class where the application's writeoperations may be performed on the data.

As indicated at 404, at a given time interval (e.g. once a month), datathat has not been modified for a given time interval (e.g. one month)may be migrated from a first, writeable, storage class to a second,read-only, storage class of the multi-class file system. In oneembodiment, to migrate the data to the second storage class, a writelock of the second storage class may be disabled during the migrationand re-enabled after the migration. The data on the second storage classis not modifiable by the application(s) while on the second storageclass. To modify data currently stored on the second storage class, thedata are first migrated to the first storage class. In one embodiment,data migrated to the second storage class may be compressed. In oneembodiment, a backup of the second storage class may be performed afterthe migration of aged data from the first storage class to the secondstorage class.

The multi-class file system may include a third, read-only storageclass. At a given time interval (e.g. once a quarter), data that has notbeen modified for a given time interval (e.g. three months) may bemigrated from the storage class to the third storage class. In oneembodiment, to migrate the data to the third storage class, a write lockof the third storage class may be disabled during the migration andre-enabled after the migration. The data on the third storage class isnot modifiable by the application(s) while the data is on the thirdstorage class. To modify data currently stored on the third storageclass, the data are first migrated to the first storage class. In oneembodiment, data migrated to the third storage class may be compressed.In one embodiment, a backup of the third storage class may be performedafter the migration of aged data from the second storage class to thethird storage class.

Note that migrated data remains in the same file system and is not movedoffline. Migration is performed by modifying file system metadata, andis transparent to the applications that access the multi-class filesystem. Path information in the file system metadata is not modifiedduring migration.

Since data is not modifiable by the applications while on a read-onlystorage class, the read-only storage class remains stable betweenmigrations of aged data to the read-only storage classes. When a portionof the data on a read-only storage class is deleted, the data blocks arenot erased or overwritten until the next migration of aged data to theread-only storage class. Deleting data on a read-only storage classmodifies the file system metadata to indicate the deletion and does noterase or overwrite the data blocks containing the deleted data. Thus, ifa portion of the data on a read-only storage class is deleted (e.g. ifan application deletes a file, at least part of which is on theread-only storage class), the portion of the data may later be restored,if desired, by modifying the file system metadata to point to the datablocks on the read-only storage class containing the portion of the datawithout requiring that the portion of the data be restored from abackup.

As indicated at 406, backups of each of the storage classes may beperformed at given time intervals using a backup mechanism. In oneembodiment, the backups of the read-only storage classes may beperformed after migrating aged data to the read-only storage classes. Inone embodiment, the backups may be image-based backups of the storagedevices in the storage classes. In one embodiment, lower storage classesare backed up less frequently than higher storage classes. For example,in a multi-class file system with three storage classes, a first,writeable storage class may be backed up once a week, a second,read-only storage class once a month (e.g. after migration of aged datafrom the first storage class), and a third storage class once everythree months (e.g. after migration of aged data from the second storageclass). The time intervals between the backups may be user-specifiedaccording to business requirements. Using this method of backing up amulti-class file system may result in reduced requirements for backupmedia (e.g. tapes) and potentially backup devices (e.g. tape drives).

In one embodiment, one or more of the lower storage classes may bewrite-locked, and the backups of the write-locked lower storage classesmay be performed without using split mirrors of the storage devices inthe write-locked lower storage classes.

In one embodiment, one or more incremental backups of the higher storageclasses including recently modified data may be performed between fullbackups of the higher storage classes. For example, a full backup of afirst, writeable storage class may be performed weekly, with incrementalbackups performed nightly. In one embodiment, recently modified portionsof files may be stored on writeable storage classes while less recentlymodified portions of the files are stored on or migrated to on read-onlystorage classes. Incremental backups of the writeable storage classesmay thus back up only the recently modified portions of the files,saving backup media space and backup time.

Note that the multi-class file system may include one or more writeablestorage classes and one or more read-only storage classes.Alternatively, all the storage classes may be writeable, or all thestorage classes may be read-only. Progressively less recently modified(or read-accessed, for read-only storage classes) data may be migrateddown in the hierarchy of storage classes, at progressively longer timeintervals, from a highest storage class to a lowest storage class.Backups of the storage classes may be performed progressively lessfrequently from the highest storage class to the lowest storage class.Accessed data may be migrated up in the hierarchy of storage classes;for modification, data on read-only storage classes is first migrated toa writeable storage class.

Performing Operations without Using Split Mirrors

Traditionally, to perform certain operations in a file system, forexample an image backup, a split mirror (hardware or software) may begenerated to provide a stable copy. When performing operations such asan image backup, creating a second copy for doing testing or batchprocessing on a database, periodic replication or certain batch jobs,file system cloning or device level split mirrors are typically used toachieve a stable copy of the storage. Using a multi-class file systemimplemented by an embodiment of the multi-class storage mechanism tosegregate data being actively modified (active data) from data notactively being modified (inactive data), such operations that normallyuse split mirrors may be performed without using split mirrors for allthe storage.

A split mirror is a stable copy broken off from a mirror image. In oneembodiment, if there is a storage class that stores inactive data, or inone embodiment a read-only storage class, and an operation is performedthat typically requires a split mirror, a split mirror of the devices inthe storage class may not be required. In one embodiment, for one ormore of the storage classes in a multi-class file system that areread-only or store inactive data, the devices in the storage classes mayhave a verifiable write lock that indicates the last time the devices inthe storage classes were written to, and an operation that wouldotherwise require a split mirror may be performed without actuallygenerating a split mirror if the write lock indicates the data on thestorage classes is stable. For example, to do an image backup using theexemplary backup mechanism 350 as illustrated in FIGS. 9-11, a splitmirror of the writeable storage class(es) (e.g. the first storage class354) may be generated, but a split mirror of the read-only storageclass(es) (e.g. the second and/or third storage classes) may not berequired. Instead, the write lock may be examined to determine if theread-only files are stable images (have not been modified).

The multi-device file system may support two or more storage devicessegmented into classes under one file system. If one or more storagedevices contain the file system metadata and/or data (files or portionsof files) that are actively modified, and one or more other storagedevices contain only inactive data (which may be read but not written towhile on the storage devices), then any operations that typically use asplit mirror may use a split mirror of only the first set of storagedevices containing active data. Other storage devices or classes maycontain data (files or portions of files) that are inactive. Note thatthese storage devices may be in the same class or in different classesof the multi-class storage system. The second set of storage devicescontains data that is read-only while stored on the storage devices, sono split mirror may be needed for the operation. As an example, a backupmechanism may use this technique to reduce split mirror storage neededfor backups and reduce the overhead of performing mirror operations. Insome embodiments, for the second set of storage classes, a mirror maynot be available to be split, so split mirrors may not be used foroperations on these storage classes. In other embodiments, mirrors maybe maintained for the second set of storage classes, but certainoperations may still be performed without using split mirrors of thesecond set of storage classes.

In one embodiment, the multi-class storage mechanism may track access ona block or extent basis within a file, so each block or extent of a filemay be located on one of the storage devices based on when it was lastwritten. Thus, blocks or extents of a file may be located on differentdevices, with actively written blocks or extents located on storagedevices containing actively written data, and read-only or inactiveblocks on other storage devices.

In one embodiment, a write lock may be implemented on a storage deviceor storage class, and an application may query to discover the last timeone of the storage devices or storage classes was enabled for write. Inone embodiment, an application may block the file system from enablingthe device for writing for the duration of the operation. This allowsthe application to make sure it has a stable image for the duration ofits operation.

In one embodiment, an application may check the write lock on a storagedevice or storage class to make sure it is enabled and not locked byanother entity before proceeding to perform an operation on the storagedevice or storage class without using a split mirror. A write lock mayindicate its current state as well as the last time the storage deviceor storage class was locked for write access. After the operation iscompleted for the storage device or storage class, the application mayagain check the write lock, and if the write lock indicates the storagedevice or storage class was write locked at any time during theoperation, the results of the operation may be considered invalid andthe operation may be retried. In one embodiment, the operation may beretried without using a split mirror. In one embodiment, if necessary ordesired, the operation may be retried using a split mirror. In oneembodiment, the write lock may be checked by the application during theoperation on the storage device or storage class to determine if thewrite lock indicates the storage device or storage class was writelocked at any time during the operation. In one embodiment, if thestorage device or storage class was write locked at any time during theoperation, the operation may be terminated and retried.

FIGS. 13 and 14 illustrate means for performing operations that requirea stable copy of storage classes in a multi-class file system withoutgenerating a split mirror of read-only storage classes. FIG. 13illustrates performing an operation on an exemplary multi-class filesystem that requires stable images of the storage classes without usingsplit mirrors for one or more storage classes according to oneembodiment. First storage class 652 of multi-class file system 650 maystore actively modified data. Second storage classes 654 and 656 maystore inactive or read-only data. For example, second storage class 654may store data migrated from first storage class 652 that has not beenmodified for a time interval (e.g. one month). Third storage class 656may store data migrated from second storage class 654 that has not beenmodified for a longer time interval (e.g. three months). In oneembodiment, the second and third storage classes are only written toduring the migration of inactive data to the storage classes. Thus, thesecond and third storage classes may be kept stable between migrations.In one embodiment, the second and third storage classes may bewrite-locked. To modify data on one of these storage classes, the datamay first be migrated to first storage class 652.

An application may need to perform an operation 670 that requires stablecopies of the storage classes in the multi-class file system 650, forexample image backups. The first storage class 652 includes data thatare actively modified; therefore, to obtain a stable copy of thisstorage class, the operation 670 may be performed on a split mirror 660of the first storage class 652. Since the second and third storageclasses store data that have not been modified for some time interval,the application can determine that the storage classes are stablecopies, and may thus perform the operation 670 directly on the storageclasses without using a split mirror of the storage classes. In oneembodiment, storage classes 654 and 656 may be write locked, and theapplication may check the write lock to determine the time of the lastupdate of the storage classes. In one embodiment, the application mayblock write access to the storage classes 654 and 656 to ensure thestorage classes are stable for the duration of operation 670. Results672 of the operation may be generated, for example image backups ifoperation 670 is a backup operation.

FIG. 14 is a flowchart of a method for performing operations in amulti-class file system without using split mirrors according to oneembodiment. As indicated at 400, a multi-class file system including ahierarchy of storage classes may be implemented on one or more storagedevices. In one embodiment, a multi-device feature of file systemsoftware (e.g. Veritas' VxFS) may be used to implement the multi-classfile system. Different architectures for the underlying structure of themulti-class file system were previously described. In one embodiment,each storage class includes one or more storage devices assigned to thestorage class according to one or more characteristics of the storageclass, e.g. performance characteristics. In one embodiment, the storageclasses may be ordered in the hierarchy of storage classes according toperformance characteristics from a highest storage class including oneor more high-performance storage devices to a lowest storage classcomprising one or more low-performance storage devices. In oneembodiment, one or more of the storage classes may be write-lockedstorage classes for storing data that is read-only while on thewrite-locked storage classes.

Actively modified data may be assigned and/or migrated to higher storageclasses and inactive or read-only data may be assigned and/or migratedto lower storage classes in the hierarchy by a multi-class storagemechanism according to a set of user-defined policies, as indicated at602. For example, a first storage class of the multi-class file systemmay store actively modified data. A second storage class may storeinactive or read-only data. For example, the second storage class maystore data migrated from the first storage class that has not beenmodified for a given time interval (e.g. one month) and is thusconsidered inactive. To modify data stored on the second storage class,the data may first be migrated to the first storage class. In oneembodiment, the second storage class is only written to during themigration of inactive data from the first storage class to the secondstorage class. Thus, the second storage class that stores inactive datamay be kept stable between migrations. There may be one or more storageclasses for active data and one or more storage classes for inactivedata. In one embodiment, file system metadata may be modified toindicate assignment or migration of the data. In one embodiment, thefile system software may migrate portions of a file to different storageclasses according to the user-defined policies. For example, activelywritten records of a database file may be stored in a higher storageclass of the multi-class file system, with inactive database recordsmigrated to a lower storage class.

An operation that requires a stable copy of the storage classes may needto be performed, e.g. by the file system software or by an applicationusing the multi-class file system. The operation may, for example, be abackup operation such as that described in FIG. 10, creating a secondcopy for doing testing or batch processing on a database, or any otheroperation that requires a stable copy. As indicated at 604, to performthe operation on a storage class that stores inactive data, such as thesecond storage class described above, since the storage class is keptstable between migrations of inactive data to the storage class, theoperation may be performed without using a split mirror of the storageclass. As indicated at 606, to perform the operation on another storageclass that stores data that is modifiable while on the storage class,such as the first storage class described above, a split mirror of theother storage class may be used.

In one embodiment using write locks, to perform the operation on aread-only storage class, a write lock of the storage class may beexamined to determine if the write-locked storage class has been writtento. If the write-locked storage class has not been written to, theoperation may be performed on the write-locked storage class withoutusing a split mirror of the write-locked storage class. In oneembodiment, the entity performing the operation may block the filesystem software from enabling the write-locked storage class for writingfor the duration of the operation.

CONCLUSION

Various embodiments may further include receiving, sending or storinginstructions and/or data implemented in accordance with the foregoingdescription upon a carrier medium. Generally speaking, a carrier mediummay include storage media or memory media such as magnetic or opticalmedia, e.g., disk or CD-ROM, volatile or non-volatile media such as RAM(e.g. SDRAM, DDR, RDRAM, SRAM, etc.), ROM, etc. As well as transmissionmedia or signals such as electrical, electromagnetic, or digitalsignals, conveyed via a communication medium such as network and/or awireless link.

The various methods as illustrated in the Figures and described hereinrepresent exemplary embodiments of methods. The methods may beimplemented in software, hardware, or a combination thereof. The orderof method may be changed, and various elements may be added, reordered,combined, omitted, modified, etc.

Various modifications and changes may be made as would be obvious to aperson skilled in the art having the benefit of this disclosure. It isintended that the invention embrace all such modifications and changesand, accordingly, the above description to be regarded in anillustrative rather than a restrictive sense.

1. A system, comprising: a processor; and a memory comprising programinstructions, wherein the program instructions are executable by theprocessor to implement: file system software configured to assign andmigrate data in a multi-class file system comprising a hierarchy ofstorage classes, wherein the migrated data remains online within themulti-class file system, wherein one or more of the storage classesstore data that is not modifiable but is readable by applications whilethe data is on the one or more storage classes, and wherein one or moreothers of the storage classes store data that is modifiable byapplications while the data is on the one or more storage classes; andan application configured to perform an operation on data stored in themulti-class file system, wherein the operation requires stable data,wherein, to perform the operation, the application is configured to:perform the operation on at least one of the one or more storage classesthat store data that is not modifiable without using a split mirror ofthe at least one of the one or more storage classes; and perform theoperation on at least one of the one or more other storage classes thatstore data that is modifiable using a split mirror of the at least oneof the one or more other storage classes.
 2. The system as recited inclaim 1, wherein the operation is a backup of the storage classes. 3.The system as recited in claim 1, wherein the one or more storageclasses that store data that is not modifiable by applications while thedata is on those storage classes are configured to be write locked, andwherein, to perform the operation on at least one of the one or morestorage classes that store data that is not modifiable without using asplit mirror of the at least one of the one or more storage classes, theapplication is further configured to, for each write-locked storageclass on which the operation is performed: examine a write lock of thewrite-locked storage class to determine if the write-locked storageclass could have been written to during the operation on that storageclass; and if said examine a write lock of the write-locked storageclass determines the storage class could have been written to during theoperation on that storage class, retry the operation for thewrite-locked storage class.
 4. The system as recited in claim 1, whereinthe one or more storage classes that store data that is not modifiableby applications while the data is on those storage classes areconfigured to be write locked, and wherein the application is furtherconfigured to block the file system software from enabling awrite-locked storage class for writing for the duration of theoperation.
 5. The system as recited in claim 1, wherein the dataincludes files or portions of files.
 6. The system as recited in claim1, wherein the data comprises one or more of application data and filesystem metadata.
 7. The system as recited in claim 1, wherein thestorage classes are ordered in the hierarchy according to one or morecharacteristics from a highest storage class to a lowest storage class.8. The system as recited in claim 7, wherein the one or morecharacteristics include one or more of performance and cost.
 9. Thesystem as recited in claim 1, wherein the storage classes are ordered inthe hierarchy of storage classes according to performancecharacteristics from a highest storage class comprising one or morehigh-performance storage devices to a lowest storage class comprisingone or more low-performance storage devices.
 10. A system, comprising: aplurality of storage devices; a host system configured to couple to theplurality of storage devices via a network, wherein the host systemcomprises: file system software configured to assign and migrate data ina multi-class file system comprising a hierarchy of storage classes,wherein migrated data remains online within the multi-class file system,wherein one or more of the storage classes store data that is notmodifiable but is readable by applications while the data is on the oneor more storage classes, and wherein one or more others of the storageclasses store data that is modifiable by applications while the data ison the one or more storage classes; and an application configured toperform an operation on data stored in the multi-class file system,wherein the operation requires stable data, wherein, to perform theoperation, the application is configured to: perform the operation on atleast one of the one or more storage classes that store data that is notmodifiable without using a split mirror of the at least one of the oneor more storage classes; and perform the operation on at least one ofthe one or more other storage classes that store data that is modifiableusing a split mirror of the at least one of the one or more otherstorage classes.
 11. The system as recited in claim 10, wherein theoperation is a backup of the storage classes.
 12. The system as recitedin claim 10, wherein the one or more storage classes that store datathat is not modifiable by applications while the data is on thosestorage classes are configured to be write locked, and wherein, toperform the operation on at least one of the one or more storage classesthat store data that is not modifiable without using a split mirror ofthe at least one of the one or more storage classes, the application isfurther configured to, for each write-locked storage class on which theoperation is performed: examine a write lock of the write-locked storageclass to determine if the write-locked storage class could have beenwritten to during the operation on that storage class; and if saidexamine a write lock of the write-locked storage class determines thestorage class could have been written to during the operation on thatstorage class, retry the operation for the write-locked storage class.13. The system as recited in claim 10, wherein the one or more storageclasses that store data that is not modifiable by applications while thedata is on those storage classes are configured to be write locked, andwherein the application is further configured to block the file systemsoftware from enabling a write-locked storage class for writing for theduration of the operation.
 14. A system, comprising: software means forassigning and migrating data in a multi-class file system comprising aplurality of storage classes and for providing access to the data in themulti-class file system to one or more applications, wherein migrateddata remains online within the multi-class file system, wherein one ormore of the storage classes store data that is not modifiable but isreadable by the applications while the data is on the one or morestorage classes, and wherein one or more others of the storage classesstore data that is modifiable by applications while the data is on theone or more storage classes; and means for performing operations thatrequire stable data on data stored in the multi-class file system,wherein said means for performing operations that require stable datacomprise: means for performing the operations on the one or more storageclasses that store data that is not modifiable without using a splitmirror of the one or more storage classes; and means for performing theoperations on the one or more other storage classes that store data thatis modifiable using a split mirror of the one or more other storageclasses.
 15. The system as recited in claim 14, further comprising meansfor blocking write access to the one or more storage classes that storedata that is not modifiable for the duration of the operations.
 16. Amethod, comprising: file system software assigning and migrating data ina multi-class file system comprising a hierarchy of storage classes,wherein migrated data remains online within the multi-class file system,wherein one or more of the storage classes store data that is notmodifiable but is readable by applications while the data is on the oneor more storage classes, and wherein one or more others of the storageclasses store data that is modifiable by applications while the data ison the one or more storage classes; and performing an operation on datastored in the multi-class file system, wherein the operation requiresstable data, wherein said performing the operation comprises: performingthe operation on at least one of the one or more storage classes thatstore data that is not modifiable without using a split mirror of the atleast one of the one or more storage classes; and performing theoperation on at least one of the one or more other storage classes thatstore data that is modifiable using a split mirror of the at least oneof the one or more other storage classes.
 17. The method as recited inclaim 16, wherein the operation is a backup of the storage classes. 18.The method as recited in claim 16, wherein the one or more storageclasses that store data that is not modifiable by applications while thedata is on those storage classes are write locked, and wherein saidperforming the operation on at least one of the one or more storageclasses that store data that is not modifiable without using a splitmirror of the at least one of the one or more storage classes comprises,for each write-locked storage class on which the operation is performed:examining a write lock of the write-locked storage class to determine ifthe write-locked storage class could have been written to during theoperation on that storage class; and if said examining a write lock ofthe write-locked storage class determines the storage class could havebeen written to during the operation on that storage class, retrying theoperation for the write-locked storage class.
 19. The method as recitedin claim 16, wherein the one or more storage classes that store datathat is not modifiable by applications while the data is on thosestorage classes are configured to be write locked, and wherein themethod further comprises blocking the file system software from enablinga write-locked storage class for writing for the duration of theoperation.
 20. The method as recited in claim 16, wherein the dataincludes files or portions of files.
 21. The method as recited in claim16, wherein the data comprises one or more of application data and filesystem metadata.
 22. The method as recited in claim 16, wherein thestorage classes are ordered in the hierarchy according to one or morecharacteristics from a highest storage class to a lowest storage class.23. The method as recited in claim 16, wherein the storage classes areordered in the hierarchy of storage classes according to performancecharacteristics from a highest storage class comprising one or morehigh-performance storage devices to a lowest storage class comprisingone or more low-performance storage devices.
 24. A computer-accessiblestorage medium, comprising program instructions, wherein the programinstructions are computer-executable to implement: assigning andmigrating data in a multi-class file system comprising a hierarchy ofstorage classes, wherein migrated data remains online within themulti-class file system, wherein one or more of the storage classesstore data that is not modifiable but is readable by applications whilethe data is on the one or more storage classes, and wherein one or moreothers of the storage classes store data that is modifiable byapplications while the data is on the one or more storage classes; andperforming an operation on data stored in the multi-class file system,wherein the operation requires stable data, wherein said performing theoperation comprises: performing the operation on at least one of the oneor more storage classes that store data that is not modifiable withoutusing a split mirror of the at least one of the one or more storageclasses; and performing the operation on at least one of the one or moreother storage classes that store data that is modifiable using a splitmirror of the at least one of the one or more other storage classes. 25.The computer-accessible storage medium as recited in claim 24, whereinthe operation is a backup of the storage classes.
 26. Thecomputer-accessible storage medium as recited in claim 24, wherein theone or more storage classes that store data that is not modifiable byapplications while the data is on those storage classes are writelocked, and wherein, in said performing the operation on at least one ofthe one or more storage classes that store data that is not modifiablewithout using a split mirror of the at least one of the one or morestorage classes, the program instructions are furthercomputer-executable to implement, for each write-locked storage class onwhich the operation is performed: examining a write lock of thewrite-locked storage class to determine if the write-locked storageclass could have been written to during the operation on that storageclass; and if said examining a write lock of the write-locked storageclass determines the storage class could have been written to during theoperation on that storage class, retrying the operation for thewrite-locked storage class.
 27. The computer-accessible storage mediumas recited in claim 24, wherein the one or more storage classes thatstore data that is not modifiable by applications while the data is onthose storage classes are configured to be write locked, and wherein theprogram instructions are further computer-executable to implementblocking file system software from enabling write access to awrite-locked storage class for the duration of the operation.
 28. Thecomputer-accessible storage medium as recited in claim 24, wherein thedata includes files or portions of files.
 29. The computer-accessiblestorage medium as recited in claim 24, wherein the data comprises one ormore of application data and file system metadata.
 30. Thecomputer-accessible storage medium as recited in claim 24, wherein thestorage classes are ordered in the hierarchy according to one or morecharacteristics from a highest storage class to a lowest storage class.31. The computer-accessible storage medium as recited in claim 24,wherein the storage classes are ordered in the hierarchy of storageclasses according to performance characteristics from a highest storageclass comprising one or more high-performance storage devices to alowest storage class comprising one or more low-performance storagedevices.